Problem Definition

PowerCo is a major gas and electricity utility that supplies to corporate, SME (Small & Medium Enterprise), and residential customers. The power-liberalization of the energy market in Europe has led to significant customer churn, especially in the SME segment.

A fair hypothesis is that price changes affect customer churn. Therefore, it is helpful to know which customers are more (or less) likely to churn at their current price, for which a good predictive model could be useful.

Moreover, for those customers that are at risk of churning, a discount might incentivize them to stay with our client. The head of the SME division is considering a 20% discount that is considered large enough to dissuade almost anyone from churning (especially those for whom price is the primary concern)

This problem is mainly a data science prediction problem. Through analyzing the data(The client data and the Prices data), understanding deeper insights it holds and modelling this data will enable us to build a classification model able to predict whether the customer is likely to churn. Our approach will be following the crisp DM model beginning with understanding the business, understanding the data, preparing the data for modelling, building our model then evaluating it and giving the final report.

Business Understanding

Let us begin with The features description in both of the datasets we will be working on in this project

Client Data Features Description

● id = client company identifier
● activity_new = category of the company’s activity
● channel_sales = code of the sales channel
● cons_12m = electricity consumption of the past 12 months
● cons_gas_12m = gas consumption of the past 12 months
● cons_last_month = electricity consumption of the last month
● date_activ = date of activation of the contract
● date_end = registered date of the end of the contract
● date_modif_prod = date of the last modification of the product
● date_renewal = date of the next contract renewal
● forecast_cons_12m = forecasted electricity consumption for next 12 months
● forecast_cons_year = forecasted electricity consumption for the next calendar year
● forecast_discount_energy = forecasted value of current discount
● forecast_meter_rent_12m = forecasted bill of meter rental for the next 2 months
● forecast_price_energy_off_peak = forecasted energy price for 1st period (off peak)
● forecast_price_energy_peak = forecasted energy price for 2nd period (peak)
● forecast_price_pow_off_peak = forecasted power price for 1st period (off peak)
● has_gas = indicated if client is also a gas client
● imp_cons = current paid consumption
● margin_gross_pow_ele = gross margin on power subscription
● margin_net_pow_ele = net margin on power subscription
● nb_prod_act = number of active products and services
● net_margin = total net margin
● num_years_antig = antiquity of the client (in number of years)
● origin_up = code of the electricity campaign the customer first subscribed to
● pow_max = subscribed power
● churn = has the client churned over the next 3 months

Prices Data Features Description

● id = client company identifier
● price_date = reference date
● price_off_peak_var = price of energy for the 1st period (off peak)
● price_peak_var = price of energy for the 2nd period (peak)
● price_mid_peak_var = price of energy for the 3rd period (mid peak)
● price_off_peak_fix = price of power for the 1st period (off peak)
● price_peak_fix = price of power for the 2nd period (peak)
● price_mid_peak_fix = price of power for the 3rd period (mid peak)

Exploratory Data Analysis

import pandas as pd
import seaborn as sns
import plotly.express as px
import matplotlib.pyplot as plt
import numpy as np
import warnings
from catboost import CatBoostClassifier, Pool
from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score, r2_score, roc_auc_score, roc_curve, confusion_matrix, plot_confusion_matrix
warnings.filterwarnings('ignore')
import missingno as msno
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import confusion_matrix
from xgboost import XGBClassifier
import lightgbm as lgb
from sklearn.svm import SVC
from sklearn.linear_model import LogisticRegression, SGDClassifier

import plotly.io as pio
import plotly.express as px

pio.templates.default = "plotly"

pd.set_option('display.max_columns',None)

Let's import our datasets

dfc = pd.read_csv("client_data.csv")
dfp = pd.read_csv("price_data.csv")

StatisticalOverview

dfc.head(50)

	id	channel_sales	cons_12m	cons_gas_12m	cons_last_month	date_activ	date_end	date_modif_prod	date_renewal	forecast_cons_12m	forecast_cons_year	forecast_discount_energy	forecast_meter_rent_12m	forecast_price_energy_off_peak	forecast_price_energy_peak	forecast_price_pow_off_peak	has_gas	imp_cons	margin_gross_pow_ele	margin_net_pow_ele	nb_prod_act	net_margin	num_years_antig	origin_up	pow_max	churn
0	24011ae4ebbe3035111d65fa7c15bc57	foosdfpfkusacimwkcsosbicdxkicaua	0	54946	0	2013-06-15	2016-06-15	2015-11-01	2015-06-23	0.00	0	0.0	1.78	0.114481	0.098142	40.606701	t	0.00	25.44	25.44	2	678.99	3	lxidpiddsbxsbosboudacockeimpuepw	43.648	1
1	d29c2c54acc38ff3c0614d0a653813dd	MISSING	4660	0	0	2009-08-21	2016-08-30	2009-08-21	2015-08-31	189.95	0	0.0	16.27	0.145711	0.000000	44.311378	f	0.00	16.38	16.38	1	18.89	6	kamkkxfxxuwbdslkwifmmcsiusiuosws	13.800	0
2	764c75f661154dac3a6c254cd082ea7d	foosdfpfkusacimwkcsosbicdxkicaua	544	0	0	2010-04-16	2016-04-16	2010-04-16	2015-04-17	47.96	0	0.0	38.72	0.165794	0.087899	44.311378	f	0.00	28.60	28.60	1	6.60	6	kamkkxfxxuwbdslkwifmmcsiusiuosws	13.856	0
3	bba03439a292a1e166f80264c16191cb	lmkebamcaaclubfxadlmueccxoimlema	1584	0	0	2010-03-30	2016-03-30	2010-03-30	2015-03-31	240.04	0	0.0	19.83	0.146694	0.000000	44.311378	f	0.00	30.22	30.22	1	25.46	6	kamkkxfxxuwbdslkwifmmcsiusiuosws	13.200	0
4	149d57cf92fc41cf94415803a877cb4b	MISSING	4425	0	526	2010-01-13	2016-03-07	2010-01-13	2015-03-09	445.75	526	0.0	131.73	0.116900	0.100015	40.606701	f	52.32	44.91	44.91	1	47.98	6	kamkkxfxxuwbdslkwifmmcsiusiuosws	19.800	0
5	1aa498825382410b098937d65c4ec26d	usilxuppasemubllopkaafesmlibmsdf	8302	0	1998	2011-12-09	2016-12-09	2015-11-01	2015-12-10	796.94	1998	0.0	30.12	0.164775	0.086131	45.308378	f	181.21	33.12	33.12	1	118.89	4	lxidpiddsbxsbosboudacockeimpuepw	13.200	1
6	7ab4bf4878d8f7661dfc20e9b8e18011	foosdfpfkusacimwkcsosbicdxkicaua	45097	0	0	2011-12-02	2016-12-02	2011-12-02	2015-12-03	8069.28	0	0.0	0.00	0.166178	0.087538	44.311378	f	0.00	4.04	4.04	1	346.63	4	lxidpiddsbxsbosboudacockeimpuepw	15.000	1
7	01495c955be7ec5e7f3203406785aae0	foosdfpfkusacimwkcsosbicdxkicaua	29552	0	1260	2010-04-21	2016-04-21	2010-04-21	2015-04-22	864.73	751	0.0	144.49	0.115174	0.098837	40.606701	f	70.63	53.92	53.92	1	100.09	6	lxidpiddsbxsbosboudacockeimpuepw	26.400	0
8	f53a254b1115634330c12c7fdbf7958a	usilxuppasemubllopkaafesmlibmsdf	2962	0	0	2011-09-23	2016-09-23	2011-09-23	2015-09-25	444.38	0	0.0	15.85	0.145711	0.000000	44.311378	f	0.00	12.82	12.82	1	42.59	4	kamkkxfxxuwbdslkwifmmcsiusiuosws	13.200	0
9	10c1b2f97a2d2a6f10299dc213d1a370	lmkebamcaaclubfxadlmueccxoimlema	26064	0	2188	2010-05-04	2016-05-04	2015-04-29	2015-05-05	2738.10	2188	0.0	130.43	0.115761	0.099419	40.606701	f	219.59	33.42	33.42	1	329.60	6	lxidpiddsbxsbosboudacockeimpuepw	31.500	0
10	d5e512dbc8118a830a7b8c7f61d89992	usilxuppasemubllopkaafesmlibmsdf	10831	0	3112	2011-07-11	2016-07-11	2011-07-11	2015-07-12	1204.58	3112	0.0	20.94	0.164637	0.087381	44.311378	f	349.48	14.52	14.52	1	108.50	5	kamkkxfxxuwbdslkwifmmcsiusiuosws	11.500	0
11	6ee77a019251bcc304c88167e0dfcb4c	foosdfpfkusacimwkcsosbicdxkicaua	4442	0	2489	2009-11-26	2016-11-26	2009-11-26	2015-11-27	561.96	2489	0.0	20.50	0.167086	0.088444	46.305378	f	308.41	21.09	21.09	1	18.41	6	kamkkxfxxuwbdslkwifmmcsiusiuosws	10.392	0
12	2c05848e1a577dd041ea6804bbf5bbf5	foosdfpfkusacimwkcsosbicdxkicaua	46121	1542867	5280	2011-09-01	2016-10-30	2013-10-30	2015-11-01	4502.43	5280	0.0	145.81	0.115237	0.100123	40.606701	t	523.68	5.52	5.52	2	533.46	4	lxidpiddsbxsbosboudacockeimpuepw	16.500	0
13	21860c2ff2d5df75503b230ce629c253	foosdfpfkusacimwkcsosbicdxkicaua	7726	0	0	2013-01-22	2017-01-22	2015-11-18	2016-01-24	914.10	0	0.0	74.66	0.162327	0.084428	44.311378	f	0.00	33.12	33.12	1	89.02	3	ldkssxwpmemidmecebumciepifcamkci	13.200	0
14	c5037389dd8be351d3c40f2973da2391	MISSING	9707	0	0	2010-12-01	2016-12-01	2010-12-01	2015-12-02	1028.52	0	0.0	15.37	0.168242	0.089602	45.308378	f	0.00	15.78	15.78	1	111.59	5	kamkkxfxxuwbdslkwifmmcsiusiuosws	13.856	0
15	e4773b60d2e68aac94795782abfc006b	lmkebamcaaclubfxadlmueccxoimlema	47019	0	3161	2012-04-20	2016-04-20	2012-04-20	2015-04-21	1687.29	1080	0.0	16.08	0.144902	0.000000	44.311378	f	158.53	19.28	19.28	1	130.14	4	ldkssxwpmemidmecebumciepifcamkci	10.392	0
16	bd14bbac3aca710ee5288fcd8511d4bb	MISSING	786536	80583	85883	2010-02-15	2016-02-15	2010-02-15	2015-02-16	13517.50	23298	0.0	16.60	0.164058	0.086163	44.311378	t	2585.38	25.68	25.68	2	1087.76	6	lxidpiddsbxsbosboudacockeimpuepw	11.400	0
17	4e4a34a6d91ff25d0fdf68733907ad44	foosdfpfkusacimwkcsosbicdxkicaua	30776	0	0	2011-08-01	2016-07-25	2014-07-25	2015-07-27	3045.16	0	0.0	118.24	0.112860	0.096521	40.606701	f	0.00	19.29	19.29	1	335.78	5	MISSING	25.000	0
18	f2b93783adecf0d0d8d60ab547bda3de	MISSING	24838	0	2877	2011-05-27	2016-05-27	2011-05-27	2015-05-31	1088.35	798	0.0	141.79	0.115174	0.098837	40.606701	f	79.36	21.57	21.57	1	117.44	5	ldkssxwpmemidmecebumciepifcamkci	15.001	0
19	47e976c5d7f1ee1fee6d60ca39fd0218	foosdfpfkusacimwkcsosbicdxkicaua	15411	0	0	2012-05-28	2016-05-28	2012-05-28	2015-05-31	194.42	0	0.0	18.31	0.145711	0.000000	44.311378	f	0.00	23.97	23.97	1	19.07	4	lxidpiddsbxsbosboudacockeimpuepw	13.000	0
20	db6ac41fb6972374f28bc6f8e43b9764	foosdfpfkusacimwkcsosbicdxkicaua	16782	0	1908	2009-10-01	2016-10-01	2009-10-01	2015-10-03	1686.67	1908	0.0	131.77	0.115237	0.100123	40.606701	t	189.79	6.60	6.60	2	203.35	6	kamkkxfxxuwbdslkwifmmcsiusiuosws	19.800	0
21	bee0c7e972180fbee50c2ea15b12965c	foosdfpfkusacimwkcsosbicdxkicaua	26987	0	4099	2011-03-04	2016-03-04	2011-03-04	2015-03-09	2761.95	4099	0.0	129.95	0.118636	0.101750	40.606701	f	413.68	47.35	47.35	1	316.53	5	lxidpiddsbxsbosboudacockeimpuepw	35.000	0
22	78014630fbac3e6aa980361d25cea748	usilxuppasemubllopkaafesmlibmsdf	20560	0	3804	2012-12-31	2016-12-31	2015-11-01	2016-01-01	3037.82	3804	0.0	0.00	0.142417	0.000000	44.311378	f	548.89	4.14	4.14	1	255.96	3	lxidpiddsbxsbosboudacockeimpuepw	11.000	0
23	5789c446141a44a9b405db35682f6eb5	foosdfpfkusacimwkcsosbicdxkicaua	12157	0	0	2013-02-27	2016-02-27	2015-12-16	2015-03-09	1418.81	0	0.0	16.42	0.160302	0.082403	44.311378	f	0.00	23.40	23.40	2	123.87	3	kamkkxfxxuwbdslkwifmmcsiusiuosws	10.392	0
24	ed5bc499fecb8eee257857fd286ebd65	MISSING	13122	0	0	2009-12-07	2016-12-07	2015-11-01	2015-12-08	1932.42	0	0.0	19.51	0.143478	0.000000	44.311378	f	0.00	3.56	3.56	1	150.70	6	kamkkxfxxuwbdslkwifmmcsiusiuosws	13.200	0
25	389bbbe70248fbeecdf9bb1bd0d1da04	foosdfpfkusacimwkcsosbicdxkicaua	3584	0	322	2010-11-17	2016-11-17	2010-11-17	2015-11-18	379.83	322	0.0	145.98	0.115237	0.100123	41.105201	f	32.61	57.72	57.72	1	38.76	5	lxidpiddsbxsbosboudacockeimpuepw	23.100	1
26	31615bef53242dfedeaf275214044ca1	MISSING	2632	0	0	2010-07-12	2016-07-12	2010-07-12	2015-07-14	394.83	0	0.0	15.98	0.145711	0.000000	44.311378	f	0.00	14.64	14.64	1	38.62	6	ldkssxwpmemidmecebumciepifcamkci	10.392	0
27	74ff037708f036de5745ce34d8d9d4df	foosdfpfkusacimwkcsosbicdxkicaua	4792	0	0	2011-12-12	2016-12-12	2015-11-01	2015-12-14	719.40	0	0.0	0.00	0.144039	0.000000	45.806878	f	0.00	19.42	19.42	1	60.00	4	lxidpiddsbxsbosboudacockeimpuepw	12.500	0
28	7287dad2f1f535d03435ea7c78b557c4	MISSING	7950	0	1100	2009-10-01	2016-10-01	2009-10-01	2015-10-02	856.81	1100	0.0	132.16	0.115237	0.100123	40.939027	f	113.82	20.00	20.00	1	94.01	6	kamkkxfxxuwbdslkwifmmcsiusiuosws	15.010	0
29	26c7bba7d51f86a16109de505bcd4f52	lmkebamcaaclubfxadlmueccxoimlema	1856956	41883	164355	2010-09-20	2016-09-20	2010-09-20	2015-09-23	3074.16	0	0.0	99.05	0.115237	0.100123	40.606701	t	0.00	43.44	43.44	2	487.42	5	ldkssxwpmemidmecebumciepifcamkci	29.000	0
30	44e826a55734d0ca5eeafcae0e991a75	foosdfpfkusacimwkcsosbicdxkicaua	324	0	0	2009-07-07	2016-07-07	2015-05-23	2015-07-09	8.32	0	0.0	1.08	0.143170	0.000000	44.311378	f	0.00	18.62	18.62	2	1.27	7	ldkssxwpmemidmecebumciepifcamkci	13.200	1
31	4bffd9d74bfb770ef7763e8e4e88dfc3	usilxuppasemubllopkaafesmlibmsdf	2103	0	0	2011-12-14	2016-08-06	2015-08-06	2015-01-05	355.17	0	30.0	19.60	0.198096	0.000000	46.305378	f	0.00	27.96	27.96	2	19.00	4	kamkkxfxxuwbdslkwifmmcsiusiuosws	10.392	0
32	2d9d0546d83c60ed3d5728d744d65c40	usilxuppasemubllopkaafesmlibmsdf	19593	0	0	2012-07-03	2016-07-03	2013-10-31	2015-07-04	2886.50	0	0.0	17.55	0.143166	0.000000	44.311378	f	0.00	17.45	17.45	1	243.99	4	lxidpiddsbxsbosboudacockeimpuepw	13.856	0
33	79d29481ee261caf08387bcef9e4346d	usilxuppasemubllopkaafesmlibmsdf	41814	0	4159	2012-10-10	2016-10-10	2014-06-26	2015-10-11	4107.43	4159	0.0	129.97	0.116509	0.101397	40.606701	f	412.54	10.16	10.16	1	477.42	3	lxidpiddsbxsbosboudacockeimpuepw	30.450	0
34	044d071a5dbc2668cddad4f2b28c91d1	MISSING	37969	0	3407	2005-03-28	2016-03-28	2015-03-09	2015-03-29	3879.98	3407	0.0	130.68	0.114604	0.098262	40.606701	f	330.30	44.64	44.64	1	407.98	11	ldkssxwpmemidmecebumciepifcamkci	33.000	0
35	af87b64523cf1f4f4a681e2cc44c8629	foosdfpfkusacimwkcsosbicdxkicaua	41214	0	3538	2011-11-04	2016-11-04	2015-11-01	2015-11-05	4294.49	3538	0.0	146.56	0.115360	0.100243	40.606701	f	352.08	3.72	3.72	1	505.47	4	lxidpiddsbxsbosboudacockeimpuepw	19.800	0
36	9403f3f9ff06ec9e49db8255ede2b098	lmkebamcaaclubfxadlmueccxoimlema	2012791	0	218619	2011-06-22	2016-06-22	2011-06-22	2015-06-23	547.35	346	0.0	16.12	0.144902	0.000000	44.311378	f	50.79	21.52	21.52	1	50.38	5	kamkkxfxxuwbdslkwifmmcsiusiuosws	13.200	0
37	51a7edcf29ed04830da6b3ee79f9e508	lmkebamcaaclubfxadlmueccxoimlema	20429	0	0	2010-03-12	2016-03-12	2010-03-12	2015-03-13	869.54	0	0.0	120.45	0.120372	0.103487	40.606701	f	0.00	60.39	60.39	1	111.64	6	kamkkxfxxuwbdslkwifmmcsiusiuosws	26.600	0
38	d9260de11c54a215fa7363f603bbeaf6	foosdfpfkusacimwkcsosbicdxkicaua	5957	0	794	2010-06-08	2016-06-08	2010-06-08	2015-06-23	510.87	315	0.0	18.47	0.145711	0.000000	44.311378	f	46.17	24.40	24.40	1	50.71	6	lxidpiddsbxsbosboudacockeimpuepw	15.000	0
39	a853b52e3992aa2df0870fe42d843a65	usilxuppasemubllopkaafesmlibmsdf	6047	0	0	2011-11-23	2016-11-23	2011-11-23	2015-11-24	913.00	0	0.0	19.54	0.147506	0.000000	45.806878	f	0.00	17.13	17.13	1	86.43	4	kamkkxfxxuwbdslkwifmmcsiusiuosws	10.392	0
40	91607f6e0ed7143ab1a5b2abc20f1c00	foosdfpfkusacimwkcsosbicdxkicaua	1938393	0	312684	2012-09-01	2016-09-01	2012-09-01	2015-09-02	3426.48	3537	0.0	51.60	0.143166	0.000000	44.311378	f	513.03	11.95	11.95	1	279.26	3	lxidpiddsbxsbosboudacockeimpuepw	13.850	0
41	757ea31873662fe77d60e8c4f736a1ec	foosdfpfkusacimwkcsosbicdxkicaua	9141	0	1592	2012-03-01	2016-09-20	2015-09-20	2014-03-03	419.22	523	0.0	16.32	0.142881	0.000000	46.305378	f	74.39	29.50	29.50	1	30.90	4	lxidpiddsbxsbosboudacockeimpuepw	13.856	0
42	d53f9011d82ed1f4850d0d312d04468b	ewpakwlliwisiwduibdlfmalxowmwpci	4259	0	0	2012-06-25	2016-07-09	2014-07-09	2015-07-12	632.17	0	0.0	16.74	0.144902	0.000000	44.311378	f	0.00	14.64	14.64	1	64.34	4	ldkssxwpmemidmecebumciepifcamkci	10.392	0
43	af47b8ea9a596c200ee39128a95738f9	lmkebamcaaclubfxadlmueccxoimlema	12199	0	0	2011-01-10	2016-02-29	2011-01-10	2015-03-09	1811.90	0	0.0	21.10	0.145877	0.000000	44.311378	f	0.00	23.40	23.40	1	156.21	5	ldkssxwpmemidmecebumciepifcamkci	10.392	0
44	f8943edfeb1783f3793d3abc6305c1e3	usilxuppasemubllopkaafesmlibmsdf	13859	0	0	2011-03-15	2016-03-15	2011-03-15	2015-03-16	8.83	0	0.0	18.26	0.146694	0.000000	44.311378	f	0.00	30.22	30.22	1	0.88	5	lxidpiddsbxsbosboudacockeimpuepw	13.200	0
45	adfec8a906dfa5c6d836231e65825539	foosdfpfkusacimwkcsosbicdxkicaua	110534	0	7546	2012-04-04	2016-04-04	2012-04-04	2015-04-05	3485.92	2797	0.0	131.87	0.116910	0.100572	40.606701	f	272.68	32.10	32.10	1	416.74	4	lxidpiddsbxsbosboudacockeimpuepw	26.610	0
46	34f2d4431b7db25cd660b098acf3d3a0	foosdfpfkusacimwkcsosbicdxkicaua	79090	0	3319	2009-08-01	2016-08-01	2015-07-12	2015-08-02	2690.38	2191	0.0	130.47	0.114025	0.097683	40.606701	f	201.40	12.36	12.36	1	311.76	6	ldkssxwpmemidmecebumciepifcamkci	19.800	0
47	cdcc2c363d5b34232082cee78776e59e	MISSING	5162	0	0	2010-11-18	2016-11-18	2010-11-18	2015-11-21	0.00	0	0.0	0.00	0.000600	0.000000	0.000000	f	0.00	24.33	24.33	1	80.30	5	lxidpiddsbxsbosboudacockeimpuepw	13.860	0
48	b96aa3a8655203318c6b853dbdb0ceb7	foosdfpfkusacimwkcsosbicdxkicaua	88178	0	6094	2010-01-15	2016-02-14	2013-09-14	2015-02-16	9070.33	6094	0.0	145.41	0.115744	0.098858	40.606701	f	611.08	32.88	32.88	1	719.37	6	lxidpiddsbxsbosboudacockeimpuepw	33.000	1
49	54a77b7ec412ca7133750d2cb2a24096	ewpakwlliwisiwduibdlfmalxowmwpci	22333	0	2237	2010-11-17	2016-11-17	2015-05-24	2015-11-18	2377.86	2237	0.0	143.04	0.115237	0.100123	40.606701	f	226.86	9.93	9.93	1	234.39	5	kamkkxfxxuwbdslkwifmmcsiusiuosws	52.500	0

dfp.head()

	id	price_date	price_off_peak_var	price_peak_var	price_mid_peak_var	price_off_peak_fix	price_peak_fix	price_mid_peak_fix
0	038af19179925da21a25619c5a24b745	2015-01-01	0.151367	0.0	0.0	44.266931	0.0	0.0
1	038af19179925da21a25619c5a24b745	2015-02-01	0.151367	0.0	0.0	44.266931	0.0	0.0
2	038af19179925da21a25619c5a24b745	2015-03-01	0.151367	0.0	0.0	44.266931	0.0	0.0
3	038af19179925da21a25619c5a24b745	2015-04-01	0.149626	0.0	0.0	44.266931	0.0	0.0
4	038af19179925da21a25619c5a24b745	2015-05-01	0.149626	0.0	0.0	44.266931	0.0	0.0

dfc.shape

(14606, 26)

dfp.shape

(193002, 8)

Our clients dataset has 14606 rows and 26 features whereas our prices dataset has 193002 rows and 8 features

dfc.duplicated(['id']).sum()

0

dfp.duplicated(["id"]).sum()

176906

len(dfc['id'].unique())

14606

len(dfp['id'].unique())

16096

dfc.describe()

	cons_12m	cons_gas_12m	cons_last_month	forecast_cons_12m	forecast_cons_year	forecast_discount_energy	forecast_meter_rent_12m	forecast_price_energy_off_peak	forecast_price_energy_peak	forecast_price_pow_off_peak	imp_cons	margin_gross_pow_ele	margin_net_pow_ele	nb_prod_act	net_margin	num_years_antig	pow_max	churn
count	1.460600e+04	1.460600e+04	14606.000000	14606.000000	14606.000000	14606.000000	14606.000000	14606.000000	14606.000000	14606.000000	14606.000000	14606.000000	14606.000000	14606.000000	14606.000000	14606.000000	14606.000000	14606.000000
mean	1.592203e+05	2.809238e+04	16090.269752	1868.614880	1399.762906	0.966726	63.086871	0.137283	0.050491	43.130056	152.786896	24.565121	24.562517	1.292346	189.264522	4.997809	18.135136	0.097152
std	5.734653e+05	1.629731e+05	64364.196422	2387.571531	3247.786255	5.108289	66.165783	0.024623	0.049037	4.485988	341.369366	20.231172	20.230280	0.709774	311.798130	1.611749	13.534743	0.296175
min	0.000000e+00	0.000000e+00	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	1.000000	0.000000	1.000000	3.300000	0.000000
25%	5.674750e+03	0.000000e+00	0.000000	494.995000	0.000000	0.000000	16.180000	0.116340	0.000000	40.606701	0.000000	14.280000	14.280000	1.000000	50.712500	4.000000	12.500000	0.000000
50%	1.411550e+04	0.000000e+00	792.500000	1112.875000	314.000000	0.000000	18.795000	0.143166	0.084138	44.311378	37.395000	21.640000	21.640000	1.000000	112.530000	5.000000	13.856000	0.000000
75%	4.076375e+04	0.000000e+00	3383.000000	2401.790000	1745.750000	0.000000	131.030000	0.146348	0.098837	44.311378	193.980000	29.880000	29.880000	1.000000	243.097500	6.000000	19.172500	0.000000
max	6.207104e+06	4.154590e+06	771203.000000	82902.830000	175375.000000	30.000000	599.310000	0.273963	0.195975	59.266378	15042.790000	374.640000	374.640000	32.000000	24570.650000	13.000000	320.000000	1.000000

dfp.describe()

	price_off_peak_var	price_peak_var	price_mid_peak_var	price_off_peak_fix	price_peak_fix	price_mid_peak_fix
count	193002.000000	193002.000000	193002.000000	193002.000000	193002.000000	193002.000000
mean	0.141027	0.054630	0.030496	43.334477	10.622875	6.409984
std	0.025032	0.049924	0.036298	5.410297	12.841895	7.773592
min	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000
25%	0.125976	0.000000	0.000000	40.728885	0.000000	0.000000
50%	0.146033	0.085483	0.000000	44.266930	0.000000	0.000000
75%	0.151635	0.101673	0.072558	44.444710	24.339581	16.226389
max	0.280700	0.229788	0.114102	59.444710	36.490692	17.458221

Data Types

dfc.info()

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 14606 entries, 0 to 14605
Data columns (total 26 columns):
 #   Column                          Non-Null Count  Dtype  
---  ------                          --------------  -----  
 0   id                              14606 non-null  object 
 1   channel_sales                   14606 non-null  object 
 2   cons_12m                        14606 non-null  int64  
 3   cons_gas_12m                    14606 non-null  int64  
 4   cons_last_month                 14606 non-null  int64  
 5   date_activ                      14606 non-null  object 
 6   date_end                        14606 non-null  object 
 7   date_modif_prod                 14606 non-null  object 
 8   date_renewal                    14606 non-null  object 
 9   forecast_cons_12m               14606 non-null  float64
 10  forecast_cons_year              14606 non-null  int64  
 11  forecast_discount_energy        14606 non-null  float64
 12  forecast_meter_rent_12m         14606 non-null  float64
 13  forecast_price_energy_off_peak  14606 non-null  float64
 14  forecast_price_energy_peak      14606 non-null  float64
 15  forecast_price_pow_off_peak     14606 non-null  float64
 16  has_gas                         14606 non-null  object 
 17  imp_cons                        14606 non-null  float64
 18  margin_gross_pow_ele            14606 non-null  float64
 19  margin_net_pow_ele              14606 non-null  float64
 20  nb_prod_act                     14606 non-null  int64  
 21  net_margin                      14606 non-null  float64
 22  num_years_antig                 14606 non-null  int64  
 23  origin_up                       14606 non-null  object 
 24  pow_max                         14606 non-null  float64
 25  churn                           14606 non-null  int64  
dtypes: float64(11), int64(7), object(8)
memory usage: 2.9+ MB

dfp.info()

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 193002 entries, 0 to 193001
Data columns (total 8 columns):
 #   Column              Non-Null Count   Dtype  
---  ------              --------------   -----  
 0   id                  193002 non-null  object 
 1   price_date          193002 non-null  object 
 2   price_off_peak_var  193002 non-null  float64
 3   price_peak_var      193002 non-null  float64
 4   price_mid_peak_var  193002 non-null  float64
 5   price_off_peak_fix  193002 non-null  float64
 6   price_peak_fix      193002 non-null  float64
 7   price_mid_peak_fix  193002 non-null  float64
dtypes: float64(6), object(2)
memory usage: 11.8+ MB

we notice that we have ids of companies in the prices dataset not present in the clients dataset. we will work on this later on

Missing Values

msno.matrix(dfc)

<AxesSubplot:>

msno.matrix(dfp)

<AxesSubplot:>

res = dfc.isin(['MISSING']).any().any()
if res:
    print("there are missing values in our dataset")
else:
    print("there are no missning values in our dataset")

there are missing values in our dataset

res = dfc.isin(['MISSING']).any()

res

id                                False
channel_sales                      True
cons_12m                          False
cons_gas_12m                      False
cons_last_month                   False
date_activ                        False
date_end                          False
date_modif_prod                   False
date_renewal                      False
forecast_cons_12m                 False
forecast_cons_year                False
forecast_discount_energy          False
forecast_meter_rent_12m           False
forecast_price_energy_off_peak    False
forecast_price_energy_peak        False
forecast_price_pow_off_peak       False
has_gas                           False
imp_cons                          False
margin_gross_pow_ele              False
margin_net_pow_ele                False
nb_prod_act                       False
net_margin                        False
num_years_antig                   False
origin_up                          True
pow_max                           False
churn                             False
dtype: bool

dfc[dfc['channel_sales']=='MISSING']

	id	channel_sales	cons_12m	cons_gas_12m	cons_last_month	date_activ	date_end	date_modif_prod	date_renewal	forecast_cons_12m	forecast_cons_year	forecast_discount_energy	forecast_meter_rent_12m	forecast_price_energy_off_peak	forecast_price_energy_peak	forecast_price_pow_off_peak	has_gas	imp_cons	margin_gross_pow_ele	margin_net_pow_ele	nb_prod_act	net_margin	num_years_antig	origin_up	pow_max	churn
1	d29c2c54acc38ff3c0614d0a653813dd	MISSING	4660	0	0	2009-08-21	2016-08-30	2009-08-21	2015-08-31	189.95	0	0.0	16.27	0.145711	0.000000	44.311378	f	0.00	16.38	16.38	1	18.89	6	kamkkxfxxuwbdslkwifmmcsiusiuosws	13.800	0
4	149d57cf92fc41cf94415803a877cb4b	MISSING	4425	0	526	2010-01-13	2016-03-07	2010-01-13	2015-03-09	445.75	526	0.0	131.73	0.116900	0.100015	40.606701	f	52.32	44.91	44.91	1	47.98	6	kamkkxfxxuwbdslkwifmmcsiusiuosws	19.800	0
14	c5037389dd8be351d3c40f2973da2391	MISSING	9707	0	0	2010-12-01	2016-12-01	2010-12-01	2015-12-02	1028.52	0	0.0	15.37	0.168242	0.089602	45.308378	f	0.00	15.78	15.78	1	111.59	5	kamkkxfxxuwbdslkwifmmcsiusiuosws	13.856	0
16	bd14bbac3aca710ee5288fcd8511d4bb	MISSING	786536	80583	85883	2010-02-15	2016-02-15	2010-02-15	2015-02-16	13517.50	23298	0.0	16.60	0.164058	0.086163	44.311378	t	2585.38	25.68	25.68	2	1087.76	6	lxidpiddsbxsbosboudacockeimpuepw	11.400	0
18	f2b93783adecf0d0d8d60ab547bda3de	MISSING	24838	0	2877	2011-05-27	2016-05-27	2011-05-27	2015-05-31	1088.35	798	0.0	141.79	0.115174	0.098837	40.606701	f	79.36	21.57	21.57	1	117.44	5	ldkssxwpmemidmecebumciepifcamkci	15.001	0
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
14585	cf8fdee748705aacc79b92eca825218f	MISSING	1185	0	0	2009-06-08	2016-06-08	2009-06-08	2015-06-12	176.56	0	0.0	7.59	0.144902	0.000000	44.311378	f	0.00	21.52	21.52	1	16.13	7	ldkssxwpmemidmecebumciepifcamkci	13.200	0
14594	9c93ea61edcaddf27333bc25178e760f	MISSING	6694	0	0	2011-12-01	2016-12-01	2015-11-01	2015-12-02	1007.35	0	0.0	15.98	0.144039	0.000000	45.308378	f	0.00	15.78	15.78	1	86.54	4	ldkssxwpmemidmecebumciepifcamkci	13.856	0
14595	a6096551a64c47cd240c356b0669f892	MISSING	11563	0	0	2010-02-08	2016-02-08	2015-12-16	2015-02-09	1384.83	0	0.0	16.16	0.164641	0.086742	44.311378	f	0.00	31.20	31.20	1	122.35	6	ldkssxwpmemidmecebumciepifcamkci	13.856	0
14598	1582ef35fbfa265e60bb3399bdebac87	MISSING	9441	0	480	2009-10-08	2016-10-08	2015-05-24	2015-10-09	983.97	480	0.0	132.11	0.115237	0.100123	40.939027	f	46.98	20.00	20.00	1	113.17	6	ldkssxwpmemidmecebumciepifcamkci	15.001	0
14605	563dde550fd624d7352f3de77c0cdfcd	MISSING	8730	0	0	2009-12-18	2016-12-17	2009-12-18	2015-12-21	762.41	0	0.0	1.07	0.167086	0.088454	45.311378	f	0.00	11.84	11.84	1	96.34	6	ldkssxwpmemidmecebumciepifcamkci	10.392	0

3725 rows × 26 columns

dfc[dfc['origin_up']=='MISSING']

	id	channel_sales	cons_12m	cons_gas_12m	cons_last_month	date_activ	date_end	date_modif_prod	date_renewal	forecast_cons_12m	forecast_cons_year	forecast_discount_energy	forecast_meter_rent_12m	forecast_price_energy_off_peak	forecast_price_energy_peak	forecast_price_pow_off_peak	has_gas	imp_cons	margin_gross_pow_ele	margin_net_pow_ele	nb_prod_act	net_margin	num_years_antig	origin_up	pow_max	churn
17	4e4a34a6d91ff25d0fdf68733907ad44	foosdfpfkusacimwkcsosbicdxkicaua	30776	0	0	2011-08-01	2016-07-25	2014-07-25	2015-07-27	3045.16	0	0.0	118.24	0.112860	0.096521	40.606701	f	0.00	19.29	19.29	1	335.78	5	MISSING	25.000	0
158	01fbea88102b46f036ff17ebe4d23459	lmkebamcaaclubfxadlmueccxoimlema	15040	4383	0	2010-05-07	2016-04-16	2015-04-15	2015-04-17	1625.53	0	0.0	116.84	0.117496	0.101155	40.606701	t	0.00	30.18	30.18	2	204.98	6	MISSING	25.000	0
357	9c4bde81b5b4913b46c28986b39be5ae	MISSING	30513	0	2294	2008-02-11	2016-02-11	2014-05-05	2015-02-12	3081.42	2294	0.0	131.43	0.116900	0.100015	40.606701	f	225.73	49.44	49.44	1	321.18	8	MISSING	33.000	0
499	f1d5af5e1fff4c8acaac81dce31028d0	MISSING	17855	315	0	2012-04-02	2016-06-01	2015-06-01	2015-04-04	658.67	0	0.0	18.37	0.199230	0.000000	45.806878	t	0.00	37.74	37.74	3	240.86	4	MISSING	12.000	0
1094	66ba1d7066c1d430cc17974387c10b25	ewpakwlliwisiwduibdlfmalxowmwpci	2501	0	0	2012-12-06	2016-12-06	2015-11-01	2015-12-07	287.81	0	0.0	0.00	0.144038	0.000000	46.305378	f	0.00	27.28	27.28	1	21.84	3	MISSING	13.856	0
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
12762	c854b26113307311c6dd2319ed1f613d	foosdfpfkusacimwkcsosbicdxkicaua	1141	0	9	2012-08-30	2016-05-08	2015-11-25	2015-05-13	98.14	9	0.0	95.87	0.112860	0.096521	40.606701	f	0.91	31.41	31.41	1	31.93	3	MISSING	17.321	0
13244	08194f6221bfc93f6b174f4f3194db42	MISSING	4	0	0	2010-10-21	2016-10-21	2010-10-21	2015-10-23	0.60	0	0.0	18.62	0.147506	0.000000	46.305378	f	0.00	28.76	28.76	1	0.04	5	MISSING	13.856	0
13314	4dbafe448d59092e174999b04b97b20f	MISSING	17493	0	1901	2010-11-05	2017-01-11	2016-01-11	2015-11-05	1590.92	1901	0.0	136.52	0.096347	0.088776	58.995952	f	159.84	0.00	0.00	1	204.66	5	MISSING	10.000	0
14228	adff89602e947091aa5adc5e39f2b146	lmkebamcaaclubfxadlmueccxoimlema	25551	0	1891	2011-09-16	2016-09-15	2011-09-16	2015-09-16	20.80	209	0.0	188.91	0.115237	0.100123	41.271364	f	19.89	72.15	72.15	1	0.11	4	MISSING	35.000	0
14352	ede19a19e735f1f24416966c93e5b251	foosdfpfkusacimwkcsosbicdxkicaua	70948	0	0	2013-01-08	2016-05-08	2014-04-28	2015-05-11	1840.29	0	0.0	13.29	0.161454	0.084198	44.311378	f	0.00	17.22	17.22	1	165.07	3	MISSING	10.392	0

64 rows × 26 columns

Our main task here is to understand the influence of the prices on the customers churn. So to understand the correlations between these factors we need to join both of the dataframes in a way that we have the id of the customer in a row followed by their info and then their prices info

Data Visualization

Let's start visualizing our data and we will begin by determining the correlations between our features to further analyze them

churn = dfc[['id', 'churn']]
churn.columns = ['Companies', 'churn']
churn_total = churn.groupby(churn['churn']).count()
churn_percentage = churn_total / churn_total.sum() * 100

fig = px.bar(churn_percentage.transpose() ,barmode='stack', title="Churn Distribution",text_auto=True)
fig.show()

df_corr = dfc.corr()
matrix = np.triu(df_corr)

fig, ax = plt.subplots(figsize=(23,10)) 
sns.heatmap(df_corr, vmin=-1, vmax=1, annot=True,ax=ax)

<AxesSubplot:>

1. we can notice a cluster of high positive and negative correlations between the prices of energy and power as these prices in the off peak, mid peak and peak depend on each other. for example we have a correlation equal to 0.97 between the prices of power mid peak and then peak. The higher mid peak prices the higher peak prices will be. On the other hand we have a negative correlation equal to -0.61 between peak price of power and off peak price of energy meaning that the higher prices of power in the peak the lower the prices of energy will be in the off peak and vice versa.
   
   
2. Looking at the cluster of the prices and the forecasts of these prices we can conclude that we have an overall highly positive correlation between these features except the energy price forcast off peak
   
   
3. On the other hand we notice also a high positive correlation between the current paid consumption and the forcast consumption of the year. a high positive correlation between the consumption of the last month and 12 month. We can also detect that the net_margin is moderately correlated to the forecast consumption 12 month which makes it accurate

Sales Channels

channel = dfc[['id', 'channel_sales', 'churn']]
channel = channel.groupby([channel['channel_sales'], channel['churn']])['id'].count().unstack(level=1).fillna(0)
channel_churn = (channel.div(channel.sum(axis=1), axis=0) * 100).sort_values(by=[1], ascending=False)

fig = px.bar(channel_churn,barmode='stack',text_auto=True)
fig.show()

Consumption

consumption = dfc[['id', 'cons_12m', 'cons_gas_12m', 'cons_last_month', 'imp_cons', 'has_gas', 'churn']]

fig = px.histogram(consumption,x="cons_12m",color='churn',barmode='stack',pattern_shape="churn")
fig.show()

fig = px.histogram(consumption,x="has_gas",color='churn',barmode='stack',pattern_shape="churn")
fig.show()

Our dataset is highly imbalanced as we have nearly 1/8 of our customers churning and 7/8 who are not overall.

fig = px.histogram(consumption,x="cons_gas_12m",color='churn',barmode='stack',pattern_shape="churn")
fig.show()

fig = px.histogram(consumption,x="cons_last_month",color='churn',barmode='stack',pattern_shape="churn")
fig.show()

fig = px.histogram(consumption,x="imp_cons",color='churn',barmode='stack',pattern_shape="churn")
fig.show()

fig=px.scatter(consumption,x='imp_cons',color="churn")
fig.show()
fig=px.scatter(consumption,x='has_gas',color="churn")
fig.show()
fig=px.scatter(consumption,x='cons_12m',color="churn")
fig.show()
fig=px.scatter(consumption,x='cons_last_month',color="churn")
fig.show()

Through these charts we can clearly notice 2 major things :

1. the consumption distribution is skewed in almost all of the features
   
2. the presence of outliers that we will handle in the next phase

px.scatter(data_frame=dfc,x="cons_12m", y="forecast_cons_12m",
             color = 'churn')

This chart showcases the inaccuracy of the forcasted consumption in the past 12 months compared to the actual consumption in the last 12 months

px.scatter(data_frame=dfc,x="imp_cons", y="forecast_cons_year",
             color = 'churn')

we can clearly see here the linearity in the paid consumption and the forcasted consumption in a year

   fig.update_layout(template='seaborn')

px.scatter(data_frame=dfc,x="num_years_antig", y="cons_12m",
             color = 'churn')

The antiquity of the customers reflect their consumption and as we can see the density of the consumption is overall located in those who have been customers for 3-6 years

Margins

margin = dfc[['id', 'margin_gross_pow_ele', 'margin_net_pow_ele', 'net_margin']]

fig = px.box(data_frame=dfc,x='margin_gross_pow_ele')
fig.show()
fig = px.box(data_frame=dfc,x='margin_net_pow_ele')
fig.show()
fig = px.box(data_frame=dfc,x='net_margin')
fig.show()

similarly to the margin we detect the presence of outliers that will be handled

Some other columns

Number of active products

others = dfc[['id', 'nb_prod_act', 'num_years_antig', 'origin_up', 'churn']]
products = others.groupby([others["nb_prod_act"],others["churn"]])["id"].count().unstack(level=1)
products_percentage = (products.div(products.sum(axis=1), axis=0)*100).sort_values(by=[1], ascending=False)

fig = px.bar(products_percentage.transpose() ,barmode='stack', title="Churn number of active products",text_auto=True)
fig.show()

Years of antiquity

years_antig = others.groupby([others["num_years_antig"],others["churn"]])["id"].count().unstack(level=1)
years_antig_percentage = (years_antig.div(years_antig.sum(axis=1), axis=0)*100)

fig = px.bar(years_antig_percentage.transpose() ,barmode='stack', title="Churn number of active products",text_auto=True)
fig.show()

Now that we have Explored our data let's start investigating the effect of the prices on the churning and discover the pattern of the impact of price / churning
for that we will group the prices ( of power : fix / energy : var ) accordignly to the date through creating three dataframes :
mean_year: the average prices in the last 12 months
mean_6m : the average prices in the last 6 months
mean_3m : the average price in the last 3 months
We will then merge the three dataframes on the ID of the client to have the final price dataframe

# let's convert the date columns to valid date columns
dfc["date_activ"] = pd.to_datetime(dfc["date_activ"], format='%Y-%m-%d')
dfc["date_end"] = pd.to_datetime(dfc["date_end"], format='%Y-%m-%d')
dfc["date_modif_prod"] = pd.to_datetime(dfc["date_modif_prod"], format='%Y-%m-%d')
dfc["date_renewal"] = pd.to_datetime(dfc["date_renewal"], format='%Y-%m-%d')
dfp['price_date'] = pd.to_datetime(dfp['price_date'], format='%Y-%m-%d')

# Now let us create the mean data frames
mean_year = dfp.groupby(['id']).mean().reset_index()
mean_6m = dfp[dfp['price_date'] > '2015-06-01'].groupby(['id']).mean().reset_index()
mean_3m = dfp[dfp['price_date'] > '2015-10-01'].groupby(['id']).mean().reset_index()

# moving onto combining into single dataframe
mean_year = mean_year.rename(
    index=str, 
    columns={
        "price_off_peak_var": "mean_year_price_p1_var",
        "price_peak_var": "mean_year_price_p2_var",
        "price_mid_peak_var": "mean_year_price_p3_var",
        "price_off_peak_fix": "mean_year_price_p1_fix",
        "price_peak_fix": "mean_year_price_p2_fix",
        "price_mid_peak_fix": "mean_year_price_p3_fix"
    }
)

mean_year["mean_year_price_p1"] = mean_year["mean_year_price_p1_var"] + mean_year["mean_year_price_p1_fix"]
mean_year["mean_year_price_p2"] = mean_year["mean_year_price_p2_var"] + mean_year["mean_year_price_p2_fix"]
mean_year["mean_year_price_p3"] = mean_year["mean_year_price_p3_var"] + mean_year["mean_year_price_p3_fix"]

mean_6m = mean_6m.rename(
    index=str, 
    columns={
        "price_off_peak_var": "mean_6m_price_p1_var",
        "price_peak_var": "mean_6m_price_p2_var",
        "price_mid_peak_var": "mean_6m_price_p3_var",
        "price_off_peak_fix": "mean_6m_price_p1_fix",
        "price_peak_fix": "mean_6m_price_p2_fix",
        "price_mid_peak_fix": "mean_6m_price_p3_fix"
    }
)
mean_6m["mean_6m_price_p1"] = mean_6m["mean_6m_price_p1_var"] + mean_6m["mean_6m_price_p1_fix"]
mean_6m["mean_6m_price_p2"] = mean_6m["mean_6m_price_p2_var"] + mean_6m["mean_6m_price_p2_fix"]
mean_6m["mean_6m_price_p3"] = mean_6m["mean_6m_price_p3_var"] + mean_6m["mean_6m_price_p3_fix"]

mean_3m = mean_3m.rename(
    index=str, 
    columns={
        "price_off_peak_var": "mean_3m_price_p1_var",
        "price_peak_var": "mean_3m_price_p2_var",
        "price_mid_peak_var": "mean_3m_price_p3_var",
        "price_off_peak_fix": "mean_3m_price_p1_fix",
        "price_peak_fix": "mean_3m_price_p2_fix",
        "price_mid_peak_fix": "mean_3m_price_p3_fix"
    }
)
mean_3m["mean_3m_price_p1"] = mean_3m["mean_3m_price_p1_var"] + mean_3m["mean_3m_price_p1_fix"]
mean_3m["mean_3m_price_p2"] = mean_3m["mean_3m_price_p2_var"] + mean_3m["mean_3m_price_p2_fix"]
mean_3m["mean_3m_price_p3"] = mean_3m["mean_3m_price_p3_var"] + mean_3m["mean_3m_price_p3_fix"]

# Now let's merge them into a final dataframe
price_features = pd.merge(mean_year, mean_6m, on='id')
price_features = pd.merge(price_features, mean_3m, on='id')

price_features

	id	mean_year_price_p1_var	mean_year_price_p2_var	mean_year_price_p3_var	mean_year_price_p1_fix	mean_year_price_p2_fix	mean_year_price_p3_fix	mean_year_price_p1	mean_year_price_p2	mean_year_price_p3	mean_6m_price_p1_var	mean_6m_price_p2_var	mean_6m_price_p3_var	mean_6m_price_p1_fix	mean_6m_price_p2_fix	mean_6m_price_p3_fix	mean_6m_price_p1	mean_6m_price_p2	mean_6m_price_p3	mean_3m_price_p1_var	mean_3m_price_p2_var	mean_3m_price_p3_var	mean_3m_price_p1_fix	mean_3m_price_p2_fix	mean_3m_price_p3_fix	mean_3m_price_p1	mean_3m_price_p2	mean_3m_price_p3
0	0002203ffbb812588b632b9e628cc38d	0.124338	0.103794	0.073160	40.701732	24.421038	16.280694	40.826071	24.524832	16.353854	0.121266	0.102368	0.073728	40.728885	24.437330	16.291555	40.850151	24.539698	16.365283	0.119906	0.101673	0.073719	40.728885	24.43733	16.291555	40.848791	24.539003	16.365274
1	0004351ebdd665e6ee664792efc4fd13	0.146426	0.000000	0.000000	44.385450	0.000000	0.000000	44.531877	0.000000	0.000000	0.144687	0.000000	0.000000	44.444710	0.000000	0.000000	44.589397	0.000000	0.000000	0.143943	0.000000	0.000000	44.444710	0.00000	0.000000	44.588653	0.000000	0.000000
2	0010bcc39e42b3c2131ed2ce55246e3c	0.181558	0.000000	0.000000	45.319710	0.000000	0.000000	45.501268	0.000000	0.000000	0.202024	0.000000	0.000000	45.944710	0.000000	0.000000	46.146734	0.000000	0.000000	0.201280	0.000000	0.000000	45.944710	0.00000	0.000000	46.145990	0.000000	0.000000
3	0010ee3855fdea87602a5b7aba8e42de	0.118757	0.098292	0.069032	40.647427	24.388455	16.258971	40.766185	24.486748	16.328003	0.114428	0.096080	0.069418	40.728885	24.437330	16.291555	40.843313	24.533410	16.360973	0.113068	0.095385	0.069409	40.728885	24.43733	16.291555	40.841953	24.532715	16.360964
4	00114d74e963e47177db89bc70108537	0.147926	0.000000	0.000000	44.266930	0.000000	0.000000	44.414856	0.000000	0.000000	0.146184	0.000000	0.000000	44.266930	0.000000	0.000000	44.413114	0.000000	0.000000	0.145440	0.000000	0.000000	44.266930	0.00000	0.000000	44.412370	0.000000	0.000000
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
16090	ffef185810e44254c3a4c6395e6b4d8a	0.138863	0.115125	0.080780	40.896427	24.637456	16.507972	41.035291	24.752581	16.588752	0.113848	0.095499	0.068838	40.728885	24.437330	16.291555	40.842733	24.532829	16.360393	0.112488	0.094804	0.068829	40.728885	24.43733	16.291555	40.841373	24.532134	16.360384
16091	fffac626da707b1b5ab11e8431a4d0a2	0.147137	0.000000	0.000000	44.311375	0.000000	0.000000	44.458512	0.000000	0.000000	0.145449	0.000000	0.000000	44.355820	0.000000	0.000000	44.501269	0.000000	0.000000	0.145047	0.000000	0.000000	44.444710	0.00000	0.000000	44.589757	0.000000	0.000000
16092	fffc0cacd305dd51f316424bbb08d1bd	0.153879	0.129497	0.094842	41.160171	24.895768	16.763569	41.314049	25.025265	16.858411	0.152759	0.127566	0.091403	41.228885	24.937330	16.791555	41.381645	25.064896	16.882958	0.151399	0.126871	0.091394	41.228885	24.93733	16.791555	41.380284	25.064201	16.882949
16093	fffe4f5646aa39c7f97f95ae2679ce64	0.123858	0.103499	0.073735	40.606699	24.364017	16.242678	40.730558	24.467516	16.316414	0.120150	0.101570	0.073983	40.647429	24.388454	16.258969	40.767579	24.490024	16.332952	0.118175	0.100491	0.074516	40.728885	24.43733	16.291555	40.847060	24.537821	16.366071
16094	ffff7fa066f1fb305ae285bb03bf325a	0.125360	0.104895	0.075635	40.647427	24.388455	16.258971	40.772788	24.493350	16.334606	0.121276	0.102927	0.076266	40.728885	24.437330	16.291555	40.850161	24.540257	16.367821	0.119916	0.102232	0.076257	40.728885	24.43733	16.291555	40.848801	24.539562	16.367812

16095 rows × 28 columns

# adding the churn column and merging with the ID
price_analysis = pd.merge(price_features, dfc[['id', 'churn']], on='id')
price_analysis.head()

	id	mean_year_price_p1_var	mean_year_price_p2_var	mean_year_price_p3_var	mean_year_price_p1_fix	mean_year_price_p2_fix	mean_year_price_p3_fix	mean_year_price_p1	mean_year_price_p2	mean_year_price_p3	mean_6m_price_p1_var	mean_6m_price_p2_var	mean_6m_price_p3_var	mean_6m_price_p1_fix	mean_6m_price_p2_fix	mean_6m_price_p3_fix	mean_6m_price_p1	mean_6m_price_p2	mean_6m_price_p3	mean_3m_price_p1_var	mean_3m_price_p2_var	mean_3m_price_p3_var	mean_3m_price_p1_fix	mean_3m_price_p2_fix	mean_3m_price_p3_fix	mean_3m_price_p1	mean_3m_price_p2	mean_3m_price_p3	churn
0	0002203ffbb812588b632b9e628cc38d	0.124338	0.103794	0.073160	40.701732	24.421038	16.280694	40.826071	24.524832	16.353854	0.121266	0.102368	0.073728	40.728885	24.43733	16.291555	40.850151	24.539698	16.365283	0.119906	0.101673	0.073719	40.728885	24.43733	16.291555	40.848791	24.539003	16.365274	0
1	0004351ebdd665e6ee664792efc4fd13	0.146426	0.000000	0.000000	44.385450	0.000000	0.000000	44.531877	0.000000	0.000000	0.144687	0.000000	0.000000	44.444710	0.00000	0.000000	44.589397	0.000000	0.000000	0.143943	0.000000	0.000000	44.444710	0.00000	0.000000	44.588653	0.000000	0.000000	0
2	0010bcc39e42b3c2131ed2ce55246e3c	0.181558	0.000000	0.000000	45.319710	0.000000	0.000000	45.501268	0.000000	0.000000	0.202024	0.000000	0.000000	45.944710	0.00000	0.000000	46.146734	0.000000	0.000000	0.201280	0.000000	0.000000	45.944710	0.00000	0.000000	46.145990	0.000000	0.000000	0
3	00114d74e963e47177db89bc70108537	0.147926	0.000000	0.000000	44.266930	0.000000	0.000000	44.414856	0.000000	0.000000	0.146184	0.000000	0.000000	44.266930	0.00000	0.000000	44.413114	0.000000	0.000000	0.145440	0.000000	0.000000	44.266930	0.00000	0.000000	44.412370	0.000000	0.000000	0
4	0013f326a839a2f6ad87a1859952d227	0.126076	0.105542	0.074921	40.728885	24.437330	16.291555	40.854961	24.542872	16.366476	0.123007	0.104108	0.075469	40.728885	24.43733	16.291555	40.851892	24.541438	16.367024	0.121647	0.103413	0.075460	40.728885	24.43733	16.291555	40.850532	24.540743	16.367015	0

let's explore the relationship between some of the price features from the meanyear the mean

fig = px.scatter(price_analysis, x="mean_year_price_p1", y="churn", marginal_x="box", marginal_y="violin")
fig.show()

fig = px.scatter(price_analysis, x="mean_6m_price_p2", y="churn", marginal_x="box", marginal_y="violin")
fig.show()

fig = px.scatter(price_analysis, x="mean_3m_price_p3", y="churn", marginal_x="box", marginal_y="violin")
fig.show()

There is a very low positive correlation with the churn shown in the previous 3 jointgridss

corr = price_analysis.corr()
# Plot correlation
plt.figure(figsize=(20,18))
sns.heatmap(corr, xticklabels=corr.columns.values, yticklabels=corr.columns.values, annot = True, annot_kws={'size':10})
# Axis ticks size
plt.xticks(fontsize=10)
plt.yticks(fontsize=10)
plt.show()

Looking at the heat map we can clearly see the non correlation between the the price features and the churn feature and thus we can conclude the absence of effect of the price on churning of customers

Data Preparation

Since the pricess have no effect on the churning of the customers we will discard the prices dataframe and focus on the client dataframe and do the necassary transformations to prepare the data for our model

our approach will be like the following :

• Data transformation
• Outliers Handling
• Data imputation
• Feature Encoding
• Feature Engineering
• Feature Normalization
• Feature Selection

Data Transformation

As shown in the exploratory data analysis phase our numerical features are highly skewed . We need to handle this skewness since most of the predictive models typically assume that the data is independent and normally distributed.
Some of the techniques to use for skewness handling are Square root - Cubic root - Logarithm. in our case we will use the logarithm technique for positive high skewness

#extracting the skewed columns
skewed_columns = [
'cons_12m',
'cons_gas_12m',
'cons_last_month',
'forecast_cons_12m',
'forecast_cons_year',
'forecast_discount_energy',
'forecast_meter_rent_12m',
'forecast_price_energy_off_peak',
'forecast_price_energy_peak',
'forecast_price_pow_off_peak'
]

# Apply inglog10 transformation
dfc["cons_12m"] = np.sqrt(dfc["cons_12m"] )
dfc["cons_gas_12m"] = np.sqrt(dfc["cons_gas_12m"] )
dfc["cons_last_month"] = np.sqrt(dfc["cons_last_month"] )
dfc["forecast_cons_12m"] = np.sqrt(dfc["forecast_cons_12m"] )
dfc["forecast_cons_year"] = np.sqrt(dfc["forecast_cons_year"] )
dfc["forecast_meter_rent_12m"] = np.sqrt(dfc["forecast_meter_rent_12m"])
dfc["imp_cons"] = np.sqrt(dfc["imp_cons"] )

Now let's see some of the new distributions

dfc

	id	channel_sales	cons_12m	cons_gas_12m	cons_last_month	date_activ	date_end	date_modif_prod	date_renewal	forecast_cons_12m	forecast_cons_year	forecast_discount_energy	forecast_meter_rent_12m	forecast_price_energy_off_peak	forecast_price_energy_peak	forecast_price_pow_off_peak	has_gas	imp_cons	margin_gross_pow_ele	margin_net_pow_ele	nb_prod_act	net_margin	num_years_antig	origin_up	pow_max	churn
0	24011ae4ebbe3035111d65fa7c15bc57	foosdfpfkusacimwkcsosbicdxkicaua	0.000000	234.405631	0.000000	2013-06-15	2016-06-15	2015-11-01	2015-06-23	0.000000	0.000000	0.0	1.334166	0.114481	0.098142	40.606701	t	0.000000	25.44	25.44	2	678.99	3	lxidpiddsbxsbosboudacockeimpuepw	43.648	1
1	d29c2c54acc38ff3c0614d0a653813dd	MISSING	68.264193	0.000000	0.000000	2009-08-21	2016-08-30	2009-08-21	2015-08-31	13.782235	0.000000	0.0	4.033609	0.145711	0.000000	44.311378	f	0.000000	16.38	16.38	1	18.89	6	kamkkxfxxuwbdslkwifmmcsiusiuosws	13.800	0
2	764c75f661154dac3a6c254cd082ea7d	foosdfpfkusacimwkcsosbicdxkicaua	23.323808	0.000000	0.000000	2010-04-16	2016-04-16	2010-04-16	2015-04-17	6.925316	0.000000	0.0	6.222540	0.165794	0.087899	44.311378	f	0.000000	28.60	28.60	1	6.60	6	kamkkxfxxuwbdslkwifmmcsiusiuosws	13.856	0
3	bba03439a292a1e166f80264c16191cb	lmkebamcaaclubfxadlmueccxoimlema	39.799497	0.000000	0.000000	2010-03-30	2016-03-30	2010-03-30	2015-03-31	15.493224	0.000000	0.0	4.453089	0.146694	0.000000	44.311378	f	0.000000	30.22	30.22	1	25.46	6	kamkkxfxxuwbdslkwifmmcsiusiuosws	13.200	0
4	149d57cf92fc41cf94415803a877cb4b	MISSING	66.520673	0.000000	22.934690	2010-01-13	2016-03-07	2010-01-13	2015-03-09	21.112792	22.934690	0.0	11.477369	0.116900	0.100015	40.606701	f	7.233257	44.91	44.91	1	47.98	6	kamkkxfxxuwbdslkwifmmcsiusiuosws	19.800	0
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
14601	18463073fb097fc0ac5d3e040f356987	foosdfpfkusacimwkcsosbicdxkicaua	179.638526	218.952050	0.000000	2012-05-24	2016-05-08	2015-05-08	2014-05-26	68.176316	0.000000	0.0	4.309292	0.138305	0.000000	44.311378	t	0.000000	27.88	27.88	2	381.77	4	lxidpiddsbxsbosboudacockeimpuepw	15.000	0
14602	d0a6f71671571ed83b2645d23af6de00	foosdfpfkusacimwkcsosbicdxkicaua	84.988234	0.000000	13.453624	2012-08-27	2016-08-27	2012-08-27	2015-08-28	25.133444	13.453624	0.0	12.001250	0.100167	0.091892	58.995952	f	3.992493	0.00	0.00	1	90.34	3	lxidpiddsbxsbosboudacockeimpuepw	6.000	1
14603	10e6828ddd62cbcf687cb74928c4c2d2	foosdfpfkusacimwkcsosbicdxkicaua	42.941821	0.000000	13.379088	2012-02-08	2016-02-07	2012-02-08	2015-02-09	13.798188	13.379088	0.0	11.384200	0.116900	0.100015	40.606701	f	4.248529	39.84	39.84	1	20.38	4	lxidpiddsbxsbosboudacockeimpuepw	15.935	1
14604	1cf20fd6206d7678d5bcafd28c53b4db	foosdfpfkusacimwkcsosbicdxkicaua	11.445523	0.000000	0.000000	2012-08-30	2016-08-30	2012-08-30	2015-08-31	4.397727	0.000000	0.0	2.679552	0.145711	0.000000	44.311378	f	0.000000	13.08	13.08	1	0.96	3	lxidpiddsbxsbosboudacockeimpuepw	11.000	0
14605	563dde550fd624d7352f3de77c0cdfcd	MISSING	93.434469	0.000000	0.000000	2009-12-18	2016-12-17	2009-12-18	2015-12-21	27.611773	0.000000	0.0	1.034408	0.167086	0.088454	45.311378	f	0.000000	11.84	11.84	1	96.34	6	ldkssxwpmemidmecebumciepifcamkci	10.392	0

14606 rows × 26 columns

 fig, axs = plt.subplots(nrows=3, figsize=(18, 20))

sns.distplot((dfc["cons_12m"].dropna()), ax=axs[0])
sns.distplot((dfc[dfc["has_gas"]=='t']["cons_gas_12m"].dropna()), ax=axs[1])
sns.distplot((dfc["cons_last_month"].dropna()), ax=axs[2])
plt.show()

This technique handled the skewness quite well ! Now we have a normal distribution almost on most of our features

Handling outliers

dfc = dfc[dfc["forecast_cons_12m"]<40000]
dfc = dfc[dfc["imp_cons"]<9000]
dfc = dfc[dfc["margin_gross_pow_ele"]<230]
dfc = dfc[dfc["margin_net_pow_ele"]<230]
dfc = dfc[dfc["net_margin"]<5000]

dfc.shape

(14600, 26)

dfc.reset_index(drop=True,inplace=True)

data imputation

channel_sales_freq = dfc['channel_sales'].value_counts()

channel_sales_freq.mode()

0       2
1       3
2      11
3     893
4    1374
5    1843
6    3724
7    6750
Name: channel_sales, dtype: int64

# we will fill the missing value with the most frequent channel_sales
dfc.loc[dfc["channel_sales"]=="MISSING","channel_sales"]="foosdfpfkusacimwkcsosbicdxkicaua"

dfc["channel_sales"].unique()

array(['foosdfpfkusacimwkcsosbicdxkicaua',
       'lmkebamcaaclubfxadlmueccxoimlema',
       'usilxuppasemubllopkaafesmlibmsdf',
       'ewpakwlliwisiwduibdlfmalxowmwpci',
       'epumfxlbckeskwekxbiuasklxalciiuu',
       'sddiedcslfslkckwlfkdpoeeailfpeds',
       'fixdbufsefwooaasfcxdxadsiekoceaa'], dtype=object)

Feature Encoding

We will start by converting the IDs into numbers

dfc

	id	channel_sales	cons_12m	cons_gas_12m	cons_last_month	date_activ	date_end	date_modif_prod	date_renewal	forecast_cons_12m	forecast_cons_year	forecast_discount_energy	forecast_meter_rent_12m	forecast_price_energy_off_peak	forecast_price_energy_peak	forecast_price_pow_off_peak	has_gas	imp_cons	margin_gross_pow_ele	margin_net_pow_ele	nb_prod_act	net_margin	num_years_antig	origin_up	pow_max	churn
0	24011ae4ebbe3035111d65fa7c15bc57	foosdfpfkusacimwkcsosbicdxkicaua	0.000000	234.405631	0.000000	2013-06-15	2016-06-15	2015-11-01	2015-06-23	0.000000	0.000000	0.0	1.334166	0.114481	0.098142	40.606701	t	0.000000	25.44	25.44	2	678.99	3	lxidpiddsbxsbosboudacockeimpuepw	43.648	1
1	d29c2c54acc38ff3c0614d0a653813dd	foosdfpfkusacimwkcsosbicdxkicaua	68.264193	0.000000	0.000000	2009-08-21	2016-08-30	2009-08-21	2015-08-31	13.782235	0.000000	0.0	4.033609	0.145711	0.000000	44.311378	f	0.000000	16.38	16.38	1	18.89	6	kamkkxfxxuwbdslkwifmmcsiusiuosws	13.800	0
2	764c75f661154dac3a6c254cd082ea7d	foosdfpfkusacimwkcsosbicdxkicaua	23.323808	0.000000	0.000000	2010-04-16	2016-04-16	2010-04-16	2015-04-17	6.925316	0.000000	0.0	6.222540	0.165794	0.087899	44.311378	f	0.000000	28.60	28.60	1	6.60	6	kamkkxfxxuwbdslkwifmmcsiusiuosws	13.856	0
3	bba03439a292a1e166f80264c16191cb	lmkebamcaaclubfxadlmueccxoimlema	39.799497	0.000000	0.000000	2010-03-30	2016-03-30	2010-03-30	2015-03-31	15.493224	0.000000	0.0	4.453089	0.146694	0.000000	44.311378	f	0.000000	30.22	30.22	1	25.46	6	kamkkxfxxuwbdslkwifmmcsiusiuosws	13.200	0
4	149d57cf92fc41cf94415803a877cb4b	foosdfpfkusacimwkcsosbicdxkicaua	66.520673	0.000000	22.934690	2010-01-13	2016-03-07	2010-01-13	2015-03-09	21.112792	22.934690	0.0	11.477369	0.116900	0.100015	40.606701	f	7.233257	44.91	44.91	1	47.98	6	kamkkxfxxuwbdslkwifmmcsiusiuosws	19.800	0
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
14595	18463073fb097fc0ac5d3e040f356987	foosdfpfkusacimwkcsosbicdxkicaua	179.638526	218.952050	0.000000	2012-05-24	2016-05-08	2015-05-08	2014-05-26	68.176316	0.000000	0.0	4.309292	0.138305	0.000000	44.311378	t	0.000000	27.88	27.88	2	381.77	4	lxidpiddsbxsbosboudacockeimpuepw	15.000	0
14596	d0a6f71671571ed83b2645d23af6de00	foosdfpfkusacimwkcsosbicdxkicaua	84.988234	0.000000	13.453624	2012-08-27	2016-08-27	2012-08-27	2015-08-28	25.133444	13.453624	0.0	12.001250	0.100167	0.091892	58.995952	f	3.992493	0.00	0.00	1	90.34	3	lxidpiddsbxsbosboudacockeimpuepw	6.000	1
14597	10e6828ddd62cbcf687cb74928c4c2d2	foosdfpfkusacimwkcsosbicdxkicaua	42.941821	0.000000	13.379088	2012-02-08	2016-02-07	2012-02-08	2015-02-09	13.798188	13.379088	0.0	11.384200	0.116900	0.100015	40.606701	f	4.248529	39.84	39.84	1	20.38	4	lxidpiddsbxsbosboudacockeimpuepw	15.935	1
14598	1cf20fd6206d7678d5bcafd28c53b4db	foosdfpfkusacimwkcsosbicdxkicaua	11.445523	0.000000	0.000000	2012-08-30	2016-08-30	2012-08-30	2015-08-31	4.397727	0.000000	0.0	2.679552	0.145711	0.000000	44.311378	f	0.000000	13.08	13.08	1	0.96	3	lxidpiddsbxsbosboudacockeimpuepw	11.000	0
14599	563dde550fd624d7352f3de77c0cdfcd	foosdfpfkusacimwkcsosbicdxkicaua	93.434469	0.000000	0.000000	2009-12-18	2016-12-17	2009-12-18	2015-12-21	27.611773	0.000000	0.0	1.034408	0.167086	0.088454	45.311378	f	0.000000	11.84	11.84	1	96.34	6	ldkssxwpmemidmecebumciepifcamkci	10.392	0

14600 rows × 26 columns

We will now be using target encoding for the channel sales feature using ordinal encoding

from sklearn.preprocessing import OrdinalEncoder
ord_enc = OrdinalEncoder()
dfc["channel_sales"] = ord_enc.fit_transform(dfc[["channel_sales"]])

ord_enc = OrdinalEncoder()
dfc["id"] = ord_enc.fit_transform(dfc[["id"]])

dfc["origin_up"] = ord_enc.fit_transform(dfc[["origin_up"]])

dfc["has_gas"] = ord_enc.fit_transform(dfc[["has_gas"]])

dfc

	id	channel_sales	cons_12m	cons_gas_12m	cons_last_month	date_activ	date_end	date_modif_prod	date_renewal	forecast_cons_12m	forecast_cons_year	forecast_discount_energy	forecast_meter_rent_12m	forecast_price_energy_off_peak	forecast_price_energy_peak	forecast_price_pow_off_peak	has_gas	imp_cons	margin_gross_pow_ele	margin_net_pow_ele	nb_prod_act	net_margin	num_years_antig	origin_up	pow_max	churn
0	2116.0	3.0	0.000000	234.405631	0.000000	2013-06-15	2016-06-15	2015-11-01	2015-06-23	0.000000	0.000000	0.0	1.334166	0.114481	0.098142	40.606701	1.0	0.000000	25.44	25.44	2	678.99	3	4.0	43.648	1
1	12004.0	3.0	68.264193	0.000000	0.000000	2009-08-21	2016-08-30	2009-08-21	2015-08-31	13.782235	0.000000	0.0	4.033609	0.145711	0.000000	44.311378	0.0	0.000000	16.38	16.38	1	18.89	6	2.0	13.800	0
2	6713.0	3.0	23.323808	0.000000	0.000000	2010-04-16	2016-04-16	2010-04-16	2015-04-17	6.925316	0.000000	0.0	6.222540	0.165794	0.087899	44.311378	0.0	0.000000	28.60	28.60	1	6.60	6	2.0	13.856	0
3	10657.0	4.0	39.799497	0.000000	0.000000	2010-03-30	2016-03-30	2010-03-30	2015-03-31	15.493224	0.000000	0.0	4.453089	0.146694	0.000000	44.311378	0.0	0.000000	30.22	30.22	1	25.46	6	2.0	13.200	0
4	1194.0	3.0	66.520673	0.000000	22.934690	2010-01-13	2016-03-07	2010-01-13	2015-03-09	21.112792	22.934690	0.0	11.477369	0.116900	0.100015	40.606701	0.0	7.233257	44.91	44.91	1	47.98	6	2.0	19.800	0
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
14595	1409.0	3.0	179.638526	218.952050	0.000000	2012-05-24	2016-05-08	2015-05-08	2014-05-26	68.176316	0.000000	0.0	4.309292	0.138305	0.000000	44.311378	1.0	0.000000	27.88	27.88	2	381.77	4	4.0	15.000	0
14596	11880.0	3.0	84.988234	0.000000	13.453624	2012-08-27	2016-08-27	2012-08-27	2015-08-28	25.133444	13.453624	0.0	12.001250	0.100167	0.091892	58.995952	0.0	3.992493	0.00	0.00	1	90.34	3	4.0	6.000	1
14597	996.0	3.0	42.941821	0.000000	13.379088	2012-02-08	2016-02-07	2012-02-08	2015-02-09	13.798188	13.379088	0.0	11.384200	0.116900	0.100015	40.606701	0.0	4.248529	39.84	39.84	1	20.38	4	4.0	15.935	1
14598	1676.0	3.0	11.445523	0.000000	0.000000	2012-08-30	2016-08-30	2012-08-30	2015-08-31	4.397727	0.000000	0.0	2.679552	0.145711	0.000000	44.311378	0.0	0.000000	13.08	13.08	1	0.96	3	4.0	11.000	0
14599	4939.0	3.0	93.434469	0.000000	0.000000	2009-12-18	2016-12-17	2009-12-18	2015-12-21	27.611773	0.000000	0.0	1.034408	0.167086	0.088454	45.311378	0.0	0.000000	11.84	11.84	1	96.34	6	3.0	10.392	0

14600 rows × 26 columns

dfc.drop([14596,14597,14598,14599],inplace=True)

Now that we have all of our data in numerical format let's conduct a further outliers handling using the Isolation Forest algorithm.

The Isolation Forest algorithm is a fast tree-based algorithm for anomaly detection. it takes advantage of the following properties :

Fewness : anomalous samples are a minority and there will only be a few of them in any dataset

Different : anomalous samples have values/attributes that are very different from those of normal samples

from sklearn.ensemble import IsolationForest

# def remove_outliers(df):
#     clf = IsolationForest(random_state=0).fit(df)
#     anomaly = clf.predict(df)
#     df['anomaly']= anomaly
#     outliers=df.loc[df['anomaly']==-1]
#     outlier_index=list(outliers.index)
#     print(df['anomaly'].value_counts())
#     df = df[df['anomaly'] != -1]
#     return df.loc[df['anomaly']== 1]

# dfc = remove_outliers(dfc) 
# dfc.drop(['anomaly'], axis=1, inplace=True)

Feature Engineering

We will follow a grouping approach for feature engineering and we will be creating the following features

1. total consumption of gs
   
2. forecasted price energy in total
   
3. contact duration = date end - date active in days
   

dfc['total_cons_12m'] = dfc['cons_12m'] + dfc['cons_gas_12m']

dfc['fore_casted_price_energy_total'] = dfc['forecast_price_energy_off_peak'] + dfc['forecast_price_energy_peak']

handling date features

def extract_date(df,date_col_name):
    #extracting year, month, day
    for i in range(len(df)):
        dates = df[date_col_name]
        date = dates[i].to_pydatetime().date()
        year = date.year
        month = date.month
        day = date.day
        
    #affecting the values to columns
        df.loc[i,'year_'+ date_col_name] = year
        df.loc[i,'month_'+ date_col_name] = month
        df.loc[i,'day_'+ date_col_name] = day
    return df

dfc = extract_date(dfc,"date_activ")
dfc = extract_date(dfc,"date_end")
dfc = extract_date(dfc,"date_modif_prod")
dfc = extract_date(dfc,"date_renewal")

dfc

	id	channel_sales	cons_12m	cons_gas_12m	cons_last_month	date_activ	date_end	date_modif_prod	date_renewal	forecast_cons_12m	forecast_cons_year	forecast_discount_energy	forecast_meter_rent_12m	forecast_price_energy_off_peak	forecast_price_energy_peak	forecast_price_pow_off_peak	has_gas	imp_cons	margin_gross_pow_ele	margin_net_pow_ele	nb_prod_act	net_margin	num_years_antig	origin_up	pow_max	churn	total_cons_12m	fore_casted_price_energy_total	year_date_activ	month_date_activ	day_date_activ	year_date_end	month_date_end	day_date_end	year_date_modif_prod	month_date_modif_prod	day_date_modif_prod	year_date_renewal	month_date_renewal	day_date_renewal
0	2116.0	3.0	0.000000	234.405631	0.000000	2013-06-15	2016-06-15	2015-11-01	2015-06-23	0.000000	0.000000	0.0	1.334166	0.114481	0.098142	40.606701	1.0	0.000000	25.44	25.44	2	678.99	3	4.0	43.648	1	234.405631	0.212623	2013.0	6.0	15.0	2016.0	6.0	15.0	2015.0	11.0	1.0	2015.0	6.0	23.0
1	12004.0	3.0	68.264193	0.000000	0.000000	2009-08-21	2016-08-30	2009-08-21	2015-08-31	13.782235	0.000000	0.0	4.033609	0.145711	0.000000	44.311378	0.0	0.000000	16.38	16.38	1	18.89	6	2.0	13.800	0	68.264193	0.145711	2009.0	8.0	21.0	2016.0	8.0	30.0	2009.0	8.0	21.0	2015.0	8.0	31.0
2	6713.0	3.0	23.323808	0.000000	0.000000	2010-04-16	2016-04-16	2010-04-16	2015-04-17	6.925316	0.000000	0.0	6.222540	0.165794	0.087899	44.311378	0.0	0.000000	28.60	28.60	1	6.60	6	2.0	13.856	0	23.323808	0.253693	2010.0	4.0	16.0	2016.0	4.0	16.0	2010.0	4.0	16.0	2015.0	4.0	17.0
3	10657.0	4.0	39.799497	0.000000	0.000000	2010-03-30	2016-03-30	2010-03-30	2015-03-31	15.493224	0.000000	0.0	4.453089	0.146694	0.000000	44.311378	0.0	0.000000	30.22	30.22	1	25.46	6	2.0	13.200	0	39.799497	0.146694	2010.0	3.0	30.0	2016.0	3.0	30.0	2010.0	3.0	30.0	2015.0	3.0	31.0
4	1194.0	3.0	66.520673	0.000000	22.934690	2010-01-13	2016-03-07	2010-01-13	2015-03-09	21.112792	22.934690	0.0	11.477369	0.116900	0.100015	40.606701	0.0	7.233257	44.91	44.91	1	47.98	6	2.0	19.800	0	66.520673	0.216915	2010.0	1.0	13.0	2016.0	3.0	7.0	2010.0	1.0	13.0	2015.0	3.0	9.0
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
14591	9912.0	1.0	91.159201	0.000000	26.172505	2012-12-18	2016-12-18	2012-12-18	2015-12-21	28.862606	26.172505	0.0	11.478676	0.115237	0.100123	40.939027	0.0	8.187185	24.02	24.02	1	102.52	3	2.0	23.100	0	91.159201	0.215360	2012.0	12.0	18.0	2016.0	12.0	18.0	2012.0	12.0	18.0	2015.0	12.0	21.0
14592	1244.0	3.0	97.164808	0.000000	21.908902	2009-10-08	2016-10-08	2015-05-24	2015-10-09	31.368296	21.908902	0.0	11.493911	0.115237	0.100123	40.939027	0.0	6.854196	20.00	20.00	1	113.17	6	3.0	15.001	0	97.164808	0.215360	2009.0	10.0	8.0	2016.0	10.0	8.0	2015.0	5.0	24.0	2015.0	10.0	9.0
14593	4089.0	3.0	134.770175	55.099909	0.000000	2010-01-26	2017-01-26	2015-11-17	2016-01-27	51.612208	0.000000	0.0	4.043513	0.143575	0.000000	44.311378	1.0	0.000000	31.20	31.20	3	254.81	6	2.0	13.856	0	189.870084	0.143575	2010.0	1.0	26.0	2017.0	1.0	26.0	2015.0	11.0	17.0	2016.0	1.0	27.0
14594	11158.0	3.0	295.205014	0.000000	106.616134	2013-02-08	2016-02-08	2013-02-08	2015-02-09	26.689511	26.702060	0.0	12.075595	0.120372	0.103487	40.606701	0.0	8.474078	66.00	66.00	1	87.14	3	4.0	26.400	0	295.205014	0.223859	2013.0	2.0	8.0	2016.0	2.0	8.0	2013.0	2.0	8.0	2015.0	2.0	9.0
14595	1409.0	3.0	179.638526	218.952050	0.000000	2012-05-24	2016-05-08	2015-05-08	2014-05-26	68.176316	0.000000	0.0	4.309292	0.138305	0.000000	44.311378	1.0	0.000000	27.88	27.88	2	381.77	4	4.0	15.000	0	398.590575	0.138305	2012.0	5.0	24.0	2016.0	5.0	8.0	2015.0	5.0	8.0	2014.0	5.0	26.0

14596 rows × 40 columns

dfc.drop(columns=['date_activ',"date_end","date_modif_prod","date_renewal"],inplace=True)

dfc

	id	channel_sales	cons_12m	cons_gas_12m	cons_last_month	forecast_cons_12m	forecast_cons_year	forecast_discount_energy	forecast_meter_rent_12m	forecast_price_energy_off_peak	forecast_price_energy_peak	forecast_price_pow_off_peak	has_gas	imp_cons	margin_gross_pow_ele	margin_net_pow_ele	nb_prod_act	net_margin	num_years_antig	origin_up	pow_max	churn	total_cons_12m	fore_casted_price_energy_total	year_date_activ	month_date_activ	day_date_activ	year_date_end	month_date_end	day_date_end	year_date_modif_prod	month_date_modif_prod	day_date_modif_prod	year_date_renewal	month_date_renewal	day_date_renewal
0	2116.0	3.0	0.000000	234.405631	0.000000	0.000000	0.000000	0.0	1.334166	0.114481	0.098142	40.606701	1.0	0.000000	25.44	25.44	2	678.99	3	4.0	43.648	1	234.405631	0.212623	2013.0	6.0	15.0	2016.0	6.0	15.0	2015.0	11.0	1.0	2015.0	6.0	23.0
1	12004.0	3.0	68.264193	0.000000	0.000000	13.782235	0.000000	0.0	4.033609	0.145711	0.000000	44.311378	0.0	0.000000	16.38	16.38	1	18.89	6	2.0	13.800	0	68.264193	0.145711	2009.0	8.0	21.0	2016.0	8.0	30.0	2009.0	8.0	21.0	2015.0	8.0	31.0
2	6713.0	3.0	23.323808	0.000000	0.000000	6.925316	0.000000	0.0	6.222540	0.165794	0.087899	44.311378	0.0	0.000000	28.60	28.60	1	6.60	6	2.0	13.856	0	23.323808	0.253693	2010.0	4.0	16.0	2016.0	4.0	16.0	2010.0	4.0	16.0	2015.0	4.0	17.0
3	10657.0	4.0	39.799497	0.000000	0.000000	15.493224	0.000000	0.0	4.453089	0.146694	0.000000	44.311378	0.0	0.000000	30.22	30.22	1	25.46	6	2.0	13.200	0	39.799497	0.146694	2010.0	3.0	30.0	2016.0	3.0	30.0	2010.0	3.0	30.0	2015.0	3.0	31.0
4	1194.0	3.0	66.520673	0.000000	22.934690	21.112792	22.934690	0.0	11.477369	0.116900	0.100015	40.606701	0.0	7.233257	44.91	44.91	1	47.98	6	2.0	19.800	0	66.520673	0.216915	2010.0	1.0	13.0	2016.0	3.0	7.0	2010.0	1.0	13.0	2015.0	3.0	9.0
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
14591	9912.0	1.0	91.159201	0.000000	26.172505	28.862606	26.172505	0.0	11.478676	0.115237	0.100123	40.939027	0.0	8.187185	24.02	24.02	1	102.52	3	2.0	23.100	0	91.159201	0.215360	2012.0	12.0	18.0	2016.0	12.0	18.0	2012.0	12.0	18.0	2015.0	12.0	21.0
14592	1244.0	3.0	97.164808	0.000000	21.908902	31.368296	21.908902	0.0	11.493911	0.115237	0.100123	40.939027	0.0	6.854196	20.00	20.00	1	113.17	6	3.0	15.001	0	97.164808	0.215360	2009.0	10.0	8.0	2016.0	10.0	8.0	2015.0	5.0	24.0	2015.0	10.0	9.0
14593	4089.0	3.0	134.770175	55.099909	0.000000	51.612208	0.000000	0.0	4.043513	0.143575	0.000000	44.311378	1.0	0.000000	31.20	31.20	3	254.81	6	2.0	13.856	0	189.870084	0.143575	2010.0	1.0	26.0	2017.0	1.0	26.0	2015.0	11.0	17.0	2016.0	1.0	27.0
14594	11158.0	3.0	295.205014	0.000000	106.616134	26.689511	26.702060	0.0	12.075595	0.120372	0.103487	40.606701	0.0	8.474078	66.00	66.00	1	87.14	3	4.0	26.400	0	295.205014	0.223859	2013.0	2.0	8.0	2016.0	2.0	8.0	2013.0	2.0	8.0	2015.0	2.0	9.0
14595	1409.0	3.0	179.638526	218.952050	0.000000	68.176316	0.000000	0.0	4.309292	0.138305	0.000000	44.311378	1.0	0.000000	27.88	27.88	2	381.77	4	4.0	15.000	0	398.590575	0.138305	2012.0	5.0	24.0	2016.0	5.0	8.0	2015.0	5.0	8.0	2014.0	5.0	26.0

14596 rows × 36 columns

feature Normalization

Now at this level we will move on to Feature normalization which is a process of scaling the data between 0 and 1 in order to facilitate the processing of the data for our modelling algorithm whilst creating our model and aim for better results. since this is numerical scaling we will only work on the numerical features only

def featureSelect(df_x):
    return df_x.drop(['churn'], axis=1), df_x['churn']

def Normalization(X, scaler="minmax"):
    if scaler.upper() == "STANDARD":
        stand = StandardScaler()
        X_s = stand.fit_transform(X)
        return X_s
    else:
        minmax = MinMaxScaler()
        X_mm = minmax.fit_transform(X)
        return X_mm

X, y = featureSelect(dfc)

X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=42, test_size=0.2)

Now let's normalize our numerical features

dfc.columns

Index(['id', 'channel_sales', 'cons_12m', 'cons_gas_12m', 'cons_last_month',
       'forecast_cons_12m', 'forecast_cons_year', 'forecast_discount_energy',
       'forecast_meter_rent_12m', 'forecast_price_energy_off_peak',
       'forecast_price_energy_peak', 'forecast_price_pow_off_peak', 'has_gas',
       'imp_cons', 'margin_gross_pow_ele', 'margin_net_pow_ele', 'nb_prod_act',
       'net_margin', 'num_years_antig', 'origin_up', 'pow_max', 'churn',
       'total_cons_12m', 'fore_casted_price_energy_total', 'year_date_activ',
       'month_date_activ', 'day_date_activ', 'year_date_end', 'month_date_end',
       'day_date_end', 'year_date_modif_prod', 'month_date_modif_prod',
       'day_date_modif_prod', 'year_date_renewal', 'month_date_renewal',
       'day_date_renewal'],
      dtype='object')

col_to_norm =[ 'id', 'channel_sales', 'cons_12m', 'cons_gas_12m', 'cons_last_month',
       'forecast_cons_12m', 'forecast_cons_year', 'forecast_discount_energy',
       'forecast_meter_rent_12m', 'forecast_price_energy_off_peak',
       'forecast_price_energy_peak', 'forecast_price_pow_off_peak', 'has_gas',
       'imp_cons', 'margin_gross_pow_ele', 'margin_net_pow_ele', 'nb_prod_act',
       'net_margin', 'num_years_antig', 'origin_up', 'pow_max',
       'total_cons_12m', 'fore_casted_price_energy_total', 'year_date_activ',
       'month_date_activ', 'day_date_activ', 'year_date_end', 'month_date_end',
       'day_date_end', 'year_date_modif_prod', 'month_date_modif_prod',
       'day_date_modif_prod', 'year_date_renewal', 'month_date_renewal',
       'day_date_renewal']

X_train[col_to_norm] = Normalization(X_train[col_to_norm])

Feature Selection

# CatBoost model definition
catboost_model = CatBoostClassifier(iterations=1000,
                                    custom_loss=['Accuracy'],
                                    loss_function='Logloss'
                                   )
train_pool = Pool(X_train,y_train)
# Fit CatBoost model
catboost_model.fit(train_pool,
                   plot=True)

MetricVisualizer(layout=Layout(align_self='stretch', height='500px'))

Learning rate set to 0.029422
0:	learn: 0.6664131	total: 24.5ms	remaining: 24.5s
1:	learn: 0.6414146	total: 34.6ms	remaining: 17.3s
2:	learn: 0.6179993	total: 44.6ms	remaining: 14.8s
3:	learn: 0.5970440	total: 54.1ms	remaining: 13.5s
4:	learn: 0.5773781	total: 63.8ms	remaining: 12.7s
5:	learn: 0.5585749	total: 73.6ms	remaining: 12.2s
6:	learn: 0.5409450	total: 83.6ms	remaining: 11.9s
7:	learn: 0.5247728	total: 93.4ms	remaining: 11.6s
8:	learn: 0.5100749	total: 103ms	remaining: 11.3s
9:	learn: 0.4963289	total: 112ms	remaining: 11.1s
10:	learn: 0.4834125	total: 121ms	remaining: 10.9s
11:	learn: 0.4711993	total: 130ms	remaining: 10.7s
12:	learn: 0.4599212	total: 139ms	remaining: 10.6s
13:	learn: 0.4495486	total: 149ms	remaining: 10.5s
14:	learn: 0.4396302	total: 158ms	remaining: 10.4s
15:	learn: 0.4308134	total: 167ms	remaining: 10.3s
16:	learn: 0.4219958	total: 176ms	remaining: 10.2s
17:	learn: 0.4146000	total: 185ms	remaining: 10.1s
18:	learn: 0.4077067	total: 195ms	remaining: 10.1s
19:	learn: 0.4010082	total: 205ms	remaining: 10s
20:	learn: 0.3949090	total: 214ms	remaining: 9.98s
21:	learn: 0.3887154	total: 223ms	remaining: 9.92s
22:	learn: 0.3835791	total: 232ms	remaining: 9.85s
23:	learn: 0.3782954	total: 241ms	remaining: 9.8s
24:	learn: 0.3734522	total: 250ms	remaining: 9.75s
25:	learn: 0.3689483	total: 259ms	remaining: 9.71s
26:	learn: 0.3648187	total: 269ms	remaining: 9.68s
27:	learn: 0.3608551	total: 278ms	remaining: 9.65s
28:	learn: 0.3575745	total: 287ms	remaining: 9.61s
29:	learn: 0.3539979	total: 296ms	remaining: 9.57s
30:	learn: 0.3508401	total: 305ms	remaining: 9.53s
31:	learn: 0.3476797	total: 314ms	remaining: 9.5s
32:	learn: 0.3447049	total: 327ms	remaining: 9.59s
33:	learn: 0.3423652	total: 336ms	remaining: 9.56s
34:	learn: 0.3396420	total: 346ms	remaining: 9.53s
35:	learn: 0.3371924	total: 355ms	remaining: 9.5s
36:	learn: 0.3352790	total: 364ms	remaining: 9.47s
37:	learn: 0.3334615	total: 373ms	remaining: 9.44s
38:	learn: 0.3314024	total: 383ms	remaining: 9.44s
39:	learn: 0.3293583	total: 393ms	remaining: 9.42s
40:	learn: 0.3279527	total: 399ms	remaining: 9.34s
41:	learn: 0.3265371	total: 408ms	remaining: 9.3s
42:	learn: 0.3245532	total: 417ms	remaining: 9.28s
43:	learn: 0.3230126	total: 426ms	remaining: 9.25s
44:	learn: 0.3218236	total: 434ms	remaining: 9.22s
45:	learn: 0.3205391	total: 443ms	remaining: 9.18s
46:	learn: 0.3193090	total: 451ms	remaining: 9.15s
47:	learn: 0.3180445	total: 460ms	remaining: 9.12s
48:	learn: 0.3171182	total: 468ms	remaining: 9.09s
49:	learn: 0.3161859	total: 477ms	remaining: 9.06s
50:	learn: 0.3152460	total: 486ms	remaining: 9.04s
51:	learn: 0.3144157	total: 494ms	remaining: 9s
52:	learn: 0.3130873	total: 502ms	remaining: 8.98s
53:	learn: 0.3119889	total: 511ms	remaining: 8.96s
54:	learn: 0.3113893	total: 520ms	remaining: 8.93s
55:	learn: 0.3103089	total: 528ms	remaining: 8.9s
56:	learn: 0.3094325	total: 537ms	remaining: 8.88s
57:	learn: 0.3089100	total: 545ms	remaining: 8.86s
58:	learn: 0.3083399	total: 554ms	remaining: 8.83s
59:	learn: 0.3076193	total: 562ms	remaining: 8.81s
60:	learn: 0.3069264	total: 571ms	remaining: 8.79s
61:	learn: 0.3060289	total: 581ms	remaining: 8.79s
62:	learn: 0.3052432	total: 591ms	remaining: 8.78s
63:	learn: 0.3046502	total: 599ms	remaining: 8.76s
64:	learn: 0.3038698	total: 609ms	remaining: 8.76s
65:	learn: 0.3032191	total: 618ms	remaining: 8.74s
66:	learn: 0.3024843	total: 627ms	remaining: 8.73s
67:	learn: 0.3017232	total: 635ms	remaining: 8.71s
68:	learn: 0.3009435	total: 644ms	remaining: 8.69s
69:	learn: 0.3006028	total: 653ms	remaining: 8.67s
70:	learn: 0.3002238	total: 661ms	remaining: 8.65s
71:	learn: 0.2995808	total: 670ms	remaining: 8.63s
72:	learn: 0.2990948	total: 679ms	remaining: 8.62s
73:	learn: 0.2986692	total: 687ms	remaining: 8.6s
74:	learn: 0.2983860	total: 696ms	remaining: 8.58s
75:	learn: 0.2980584	total: 704ms	remaining: 8.56s
76:	learn: 0.2975359	total: 713ms	remaining: 8.54s
77:	learn: 0.2971115	total: 721ms	remaining: 8.53s
78:	learn: 0.2967524	total: 730ms	remaining: 8.51s
79:	learn: 0.2963365	total: 738ms	remaining: 8.49s
80:	learn: 0.2959447	total: 747ms	remaining: 8.48s
81:	learn: 0.2957088	total: 756ms	remaining: 8.46s
82:	learn: 0.2950501	total: 764ms	remaining: 8.44s
83:	learn: 0.2946290	total: 774ms	remaining: 8.44s
84:	learn: 0.2942985	total: 784ms	remaining: 8.43s
85:	learn: 0.2938288	total: 793ms	remaining: 8.43s
86:	learn: 0.2934363	total: 803ms	remaining: 8.43s
87:	learn: 0.2931491	total: 813ms	remaining: 8.42s
88:	learn: 0.2927286	total: 823ms	remaining: 8.42s
89:	learn: 0.2924480	total: 832ms	remaining: 8.41s
90:	learn: 0.2921401	total: 841ms	remaining: 8.4s
91:	learn: 0.2916834	total: 851ms	remaining: 8.39s
92:	learn: 0.2914755	total: 861ms	remaining: 8.39s
93:	learn: 0.2911294	total: 870ms	remaining: 8.39s
94:	learn: 0.2906807	total: 879ms	remaining: 8.38s
95:	learn: 0.2902997	total: 889ms	remaining: 8.37s
96:	learn: 0.2898418	total: 898ms	remaining: 8.36s
97:	learn: 0.2896066	total: 906ms	remaining: 8.34s
98:	learn: 0.2892964	total: 915ms	remaining: 8.33s
99:	learn: 0.2890155	total: 924ms	remaining: 8.31s
100:	learn: 0.2886767	total: 933ms	remaining: 8.3s
101:	learn: 0.2884637	total: 942ms	remaining: 8.29s
102:	learn: 0.2881466	total: 950ms	remaining: 8.28s
103:	learn: 0.2877601	total: 961ms	remaining: 8.28s
104:	learn: 0.2875172	total: 970ms	remaining: 8.27s
105:	learn: 0.2872336	total: 982ms	remaining: 8.28s
106:	learn: 0.2869034	total: 994ms	remaining: 8.29s
107:	learn: 0.2866872	total: 1s	remaining: 8.29s
108:	learn: 0.2864231	total: 1.01s	remaining: 8.28s
109:	learn: 0.2861775	total: 1.02s	remaining: 8.28s
110:	learn: 0.2859996	total: 1.03s	remaining: 8.27s
111:	learn: 0.2856743	total: 1.04s	remaining: 8.26s
112:	learn: 0.2854096	total: 1.05s	remaining: 8.26s
113:	learn: 0.2851712	total: 1.06s	remaining: 8.25s
114:	learn: 0.2849008	total: 1.07s	remaining: 8.24s
115:	learn: 0.2846426	total: 1.08s	remaining: 8.23s
116:	learn: 0.2844306	total: 1.09s	remaining: 8.23s
117:	learn: 0.2843001	total: 1.1s	remaining: 8.23s
118:	learn: 0.2840639	total: 1.11s	remaining: 8.22s
119:	learn: 0.2837632	total: 1.12s	remaining: 8.21s
120:	learn: 0.2835368	total: 1.13s	remaining: 8.21s
121:	learn: 0.2831963	total: 1.14s	remaining: 8.22s
122:	learn: 0.2828464	total: 1.15s	remaining: 8.23s
123:	learn: 0.2826419	total: 1.17s	remaining: 8.23s
124:	learn: 0.2822731	total: 1.18s	remaining: 8.24s
125:	learn: 0.2820163	total: 1.19s	remaining: 8.24s
126:	learn: 0.2818654	total: 1.2s	remaining: 8.25s
127:	learn: 0.2814828	total: 1.21s	remaining: 8.26s
128:	learn: 0.2810810	total: 1.22s	remaining: 8.26s
129:	learn: 0.2808961	total: 1.23s	remaining: 8.26s
130:	learn: 0.2806967	total: 1.24s	remaining: 8.26s
131:	learn: 0.2805058	total: 1.27s	remaining: 8.35s
132:	learn: 0.2800644	total: 1.28s	remaining: 8.33s
133:	learn: 0.2798502	total: 1.29s	remaining: 8.32s
134:	learn: 0.2796590	total: 1.3s	remaining: 8.31s
135:	learn: 0.2794374	total: 1.3s	remaining: 8.3s
136:	learn: 0.2792255	total: 1.31s	remaining: 8.28s
137:	learn: 0.2790797	total: 1.32s	remaining: 8.27s
138:	learn: 0.2788662	total: 1.33s	remaining: 8.25s
139:	learn: 0.2785874	total: 1.34s	remaining: 8.24s
140:	learn: 0.2783416	total: 1.35s	remaining: 8.24s
141:	learn: 0.2781221	total: 1.36s	remaining: 8.22s
142:	learn: 0.2779112	total: 1.37s	remaining: 8.21s
143:	learn: 0.2777904	total: 1.38s	remaining: 8.2s
144:	learn: 0.2775212	total: 1.39s	remaining: 8.18s
145:	learn: 0.2773650	total: 1.4s	remaining: 8.18s
146:	learn: 0.2770180	total: 1.41s	remaining: 8.16s
147:	learn: 0.2769109	total: 1.42s	remaining: 8.15s
148:	learn: 0.2767026	total: 1.43s	remaining: 8.14s
149:	learn: 0.2764504	total: 1.44s	remaining: 8.14s
150:	learn: 0.2763071	total: 1.45s	remaining: 8.13s
151:	learn: 0.2760494	total: 1.46s	remaining: 8.12s
152:	learn: 0.2759079	total: 1.47s	remaining: 8.11s
153:	learn: 0.2757481	total: 1.48s	remaining: 8.1s
154:	learn: 0.2755830	total: 1.48s	remaining: 8.09s
155:	learn: 0.2754154	total: 1.5s	remaining: 8.09s
156:	learn: 0.2752234	total: 1.5s	remaining: 8.08s
157:	learn: 0.2749132	total: 1.51s	remaining: 8.07s
158:	learn: 0.2746735	total: 1.52s	remaining: 8.07s
159:	learn: 0.2744301	total: 1.53s	remaining: 8.06s
160:	learn: 0.2743006	total: 1.54s	remaining: 8.05s
161:	learn: 0.2741000	total: 1.55s	remaining: 8.04s
162:	learn: 0.2739301	total: 1.56s	remaining: 8.03s
163:	learn: 0.2737578	total: 1.57s	remaining: 8.03s
164:	learn: 0.2735936	total: 1.58s	remaining: 8.02s
165:	learn: 0.2734412	total: 1.59s	remaining: 8s
166:	learn: 0.2732508	total: 1.6s	remaining: 7.99s
167:	learn: 0.2730580	total: 1.61s	remaining: 7.98s
168:	learn: 0.2727310	total: 1.62s	remaining: 7.97s
169:	learn: 0.2726394	total: 1.63s	remaining: 7.95s
170:	learn: 0.2724290	total: 1.64s	remaining: 7.94s
171:	learn: 0.2722454	total: 1.65s	remaining: 7.92s
172:	learn: 0.2720432	total: 1.65s	remaining: 7.91s
173:	learn: 0.2717729	total: 1.66s	remaining: 7.89s
174:	learn: 0.2716136	total: 1.67s	remaining: 7.88s
175:	learn: 0.2713549	total: 1.68s	remaining: 7.87s
176:	learn: 0.2711867	total: 1.69s	remaining: 7.85s
177:	learn: 0.2711078	total: 1.7s	remaining: 7.84s
178:	learn: 0.2708572	total: 1.71s	remaining: 7.82s
179:	learn: 0.2706531	total: 1.71s	remaining: 7.81s
180:	learn: 0.2704917	total: 1.72s	remaining: 7.8s
181:	learn: 0.2703816	total: 1.73s	remaining: 7.79s
182:	learn: 0.2701849	total: 1.74s	remaining: 7.78s
183:	learn: 0.2699951	total: 1.75s	remaining: 7.77s
184:	learn: 0.2698861	total: 1.76s	remaining: 7.76s
185:	learn: 0.2697475	total: 1.77s	remaining: 7.74s
186:	learn: 0.2696184	total: 1.78s	remaining: 7.73s
187:	learn: 0.2693990	total: 1.79s	remaining: 7.72s
188:	learn: 0.2692326	total: 1.79s	remaining: 7.71s
189:	learn: 0.2690572	total: 1.8s	remaining: 7.7s
190:	learn: 0.2689016	total: 1.81s	remaining: 7.69s
191:	learn: 0.2687646	total: 1.82s	remaining: 7.67s
192:	learn: 0.2686327	total: 1.83s	remaining: 7.66s
193:	learn: 0.2684841	total: 1.84s	remaining: 7.66s
194:	learn: 0.2682829	total: 1.85s	remaining: 7.66s
195:	learn: 0.2680565	total: 1.86s	remaining: 7.65s
196:	learn: 0.2679174	total: 1.87s	remaining: 7.63s
197:	learn: 0.2677696	total: 1.88s	remaining: 7.62s
198:	learn: 0.2676374	total: 1.89s	remaining: 7.61s
199:	learn: 0.2673682	total: 1.9s	remaining: 7.59s
200:	learn: 0.2671539	total: 1.91s	remaining: 7.58s
201:	learn: 0.2669596	total: 1.92s	remaining: 7.57s
202:	learn: 0.2666639	total: 1.93s	remaining: 7.56s
203:	learn: 0.2665157	total: 1.94s	remaining: 7.57s
204:	learn: 0.2662877	total: 1.95s	remaining: 7.55s
205:	learn: 0.2660466	total: 1.96s	remaining: 7.54s
206:	learn: 0.2658888	total: 1.97s	remaining: 7.53s
207:	learn: 0.2657540	total: 1.97s	remaining: 7.52s
208:	learn: 0.2655301	total: 1.98s	remaining: 7.51s
209:	learn: 0.2652508	total: 1.99s	remaining: 7.5s
210:	learn: 0.2651292	total: 2s	remaining: 7.49s
211:	learn: 0.2649740	total: 2.01s	remaining: 7.48s
212:	learn: 0.2647735	total: 2.02s	remaining: 7.47s
213:	learn: 0.2646439	total: 2.03s	remaining: 7.46s
214:	learn: 0.2644881	total: 2.04s	remaining: 7.45s
215:	learn: 0.2643794	total: 2.05s	remaining: 7.43s
216:	learn: 0.2641960	total: 2.06s	remaining: 7.42s
217:	learn: 0.2640605	total: 2.07s	remaining: 7.41s
218:	learn: 0.2639739	total: 2.08s	remaining: 7.4s
219:	learn: 0.2638678	total: 2.08s	remaining: 7.39s
220:	learn: 0.2636981	total: 2.09s	remaining: 7.38s
221:	learn: 0.2634079	total: 2.1s	remaining: 7.37s
222:	learn: 0.2632822	total: 2.11s	remaining: 7.36s
223:	learn: 0.2630615	total: 2.12s	remaining: 7.35s
224:	learn: 0.2629140	total: 2.13s	remaining: 7.34s
225:	learn: 0.2627328	total: 2.14s	remaining: 7.33s
226:	learn: 0.2625235	total: 2.15s	remaining: 7.32s
227:	learn: 0.2623986	total: 2.16s	remaining: 7.31s
228:	learn: 0.2622749	total: 2.17s	remaining: 7.3s
229:	learn: 0.2621512	total: 2.18s	remaining: 7.29s
230:	learn: 0.2619507	total: 2.19s	remaining: 7.28s
231:	learn: 0.2617923	total: 2.19s	remaining: 7.27s
232:	learn: 0.2616519	total: 2.2s	remaining: 7.26s
233:	learn: 0.2615141	total: 2.21s	remaining: 7.25s
234:	learn: 0.2612284	total: 2.22s	remaining: 7.24s
235:	learn: 0.2610344	total: 2.23s	remaining: 7.23s
236:	learn: 0.2608619	total: 2.24s	remaining: 7.22s
237:	learn: 0.2606468	total: 2.25s	remaining: 7.21s
238:	learn: 0.2605100	total: 2.26s	remaining: 7.2s
239:	learn: 0.2603697	total: 2.27s	remaining: 7.19s
240:	learn: 0.2602281	total: 2.28s	remaining: 7.19s
241:	learn: 0.2600057	total: 2.29s	remaining: 7.18s
242:	learn: 0.2597499	total: 2.3s	remaining: 7.17s
243:	learn: 0.2595792	total: 2.31s	remaining: 7.16s
244:	learn: 0.2594363	total: 2.32s	remaining: 7.16s
245:	learn: 0.2592328	total: 2.33s	remaining: 7.15s
246:	learn: 0.2590418	total: 2.34s	remaining: 7.14s
247:	learn: 0.2589117	total: 2.35s	remaining: 7.13s
248:	learn: 0.2586713	total: 2.36s	remaining: 7.12s
249:	learn: 0.2584695	total: 2.37s	remaining: 7.11s
250:	learn: 0.2583647	total: 2.38s	remaining: 7.1s
251:	learn: 0.2581943	total: 2.39s	remaining: 7.09s
252:	learn: 0.2580364	total: 2.4s	remaining: 7.08s
253:	learn: 0.2578626	total: 2.41s	remaining: 7.08s
254:	learn: 0.2576786	total: 2.42s	remaining: 7.07s
255:	learn: 0.2575401	total: 2.43s	remaining: 7.06s
256:	learn: 0.2574000	total: 2.44s	remaining: 7.05s
257:	learn: 0.2572045	total: 2.45s	remaining: 7.04s
258:	learn: 0.2569965	total: 2.46s	remaining: 7.03s
259:	learn: 0.2567792	total: 2.47s	remaining: 7.02s
260:	learn: 0.2566083	total: 2.48s	remaining: 7.01s
261:	learn: 0.2564768	total: 2.48s	remaining: 7s
262:	learn: 0.2562005	total: 2.49s	remaining: 6.99s
263:	learn: 0.2560785	total: 2.5s	remaining: 6.98s
264:	learn: 0.2559453	total: 2.51s	remaining: 6.97s
265:	learn: 0.2558139	total: 2.52s	remaining: 6.96s
266:	learn: 0.2556868	total: 2.53s	remaining: 6.95s
267:	learn: 0.2555261	total: 2.54s	remaining: 6.93s
268:	learn: 0.2554123	total: 2.55s	remaining: 6.92s
269:	learn: 0.2552253	total: 2.56s	remaining: 6.91s
270:	learn: 0.2550628	total: 2.57s	remaining: 6.9s
271:	learn: 0.2548424	total: 2.58s	remaining: 6.89s
272:	learn: 0.2546929	total: 2.58s	remaining: 6.88s
273:	learn: 0.2545951	total: 2.59s	remaining: 6.87s
274:	learn: 0.2544361	total: 2.6s	remaining: 6.86s
275:	learn: 0.2542333	total: 2.61s	remaining: 6.85s
276:	learn: 0.2540391	total: 2.62s	remaining: 6.84s
277:	learn: 0.2538792	total: 2.63s	remaining: 6.83s
278:	learn: 0.2537358	total: 2.64s	remaining: 6.81s
279:	learn: 0.2535783	total: 2.65s	remaining: 6.8s
280:	learn: 0.2533659	total: 2.65s	remaining: 6.79s
281:	learn: 0.2531955	total: 2.66s	remaining: 6.78s
282:	learn: 0.2529472	total: 2.67s	remaining: 6.77s
283:	learn: 0.2528308	total: 2.68s	remaining: 6.76s
284:	learn: 0.2527410	total: 2.69s	remaining: 6.75s
285:	learn: 0.2525930	total: 2.7s	remaining: 6.74s
286:	learn: 0.2524116	total: 2.71s	remaining: 6.73s
287:	learn: 0.2522303	total: 2.72s	remaining: 6.72s
288:	learn: 0.2521253	total: 2.73s	remaining: 6.71s
289:	learn: 0.2520249	total: 2.73s	remaining: 6.7s
290:	learn: 0.2519010	total: 2.74s	remaining: 6.68s
291:	learn: 0.2517152	total: 2.75s	remaining: 6.67s
292:	learn: 0.2515781	total: 2.76s	remaining: 6.66s
293:	learn: 0.2514403	total: 2.77s	remaining: 6.65s
294:	learn: 0.2513111	total: 2.78s	remaining: 6.64s
295:	learn: 0.2511367	total: 2.79s	remaining: 6.63s
296:	learn: 0.2509403	total: 2.79s	remaining: 6.62s
297:	learn: 0.2507840	total: 2.8s	remaining: 6.6s
298:	learn: 0.2506522	total: 2.81s	remaining: 6.59s
299:	learn: 0.2505631	total: 2.82s	remaining: 6.58s
300:	learn: 0.2504116	total: 2.83s	remaining: 6.57s
301:	learn: 0.2502512	total: 2.84s	remaining: 6.56s
302:	learn: 0.2501482	total: 2.85s	remaining: 6.55s
303:	learn: 0.2499423	total: 2.86s	remaining: 6.54s
304:	learn: 0.2498554	total: 2.87s	remaining: 6.53s
305:	learn: 0.2496795	total: 2.87s	remaining: 6.52s
306:	learn: 0.2494892	total: 2.88s	remaining: 6.51s
307:	learn: 0.2493504	total: 2.89s	remaining: 6.5s
308:	learn: 0.2491925	total: 2.9s	remaining: 6.49s
309:	learn: 0.2490176	total: 2.91s	remaining: 6.48s
310:	learn: 0.2488403	total: 2.92s	remaining: 6.47s
311:	learn: 0.2486559	total: 2.93s	remaining: 6.46s
312:	learn: 0.2484295	total: 2.94s	remaining: 6.45s
313:	learn: 0.2482814	total: 2.94s	remaining: 6.43s
314:	learn: 0.2481329	total: 2.95s	remaining: 6.42s
315:	learn: 0.2479586	total: 2.96s	remaining: 6.41s
316:	learn: 0.2477446	total: 2.97s	remaining: 6.4s
317:	learn: 0.2475861	total: 2.99s	remaining: 6.41s
318:	learn: 0.2474691	total: 3s	remaining: 6.4s
319:	learn: 0.2472611	total: 3.01s	remaining: 6.39s
320:	learn: 0.2470600	total: 3.02s	remaining: 6.38s
321:	learn: 0.2469524	total: 3.02s	remaining: 6.37s
322:	learn: 0.2468045	total: 3.03s	remaining: 6.36s
323:	learn: 0.2466668	total: 3.04s	remaining: 6.34s
324:	learn: 0.2465277	total: 3.05s	remaining: 6.33s
325:	learn: 0.2463919	total: 3.06s	remaining: 6.32s
326:	learn: 0.2462150	total: 3.07s	remaining: 6.31s
327:	learn: 0.2460259	total: 3.08s	remaining: 6.3s
328:	learn: 0.2458504	total: 3.08s	remaining: 6.29s
329:	learn: 0.2456471	total: 3.09s	remaining: 6.28s
330:	learn: 0.2455038	total: 3.1s	remaining: 6.27s
331:	learn: 0.2452747	total: 3.11s	remaining: 6.26s
332:	learn: 0.2451686	total: 3.12s	remaining: 6.25s
333:	learn: 0.2450011	total: 3.13s	remaining: 6.24s
334:	learn: 0.2448277	total: 3.14s	remaining: 6.23s
335:	learn: 0.2446500	total: 3.15s	remaining: 6.22s
336:	learn: 0.2445633	total: 3.15s	remaining: 6.21s
337:	learn: 0.2444241	total: 3.16s	remaining: 6.2s
338:	learn: 0.2442523	total: 3.17s	remaining: 6.19s
339:	learn: 0.2441404	total: 3.18s	remaining: 6.17s
340:	learn: 0.2440518	total: 3.19s	remaining: 6.16s
341:	learn: 0.2439003	total: 3.2s	remaining: 6.15s
342:	learn: 0.2437246	total: 3.21s	remaining: 6.14s
343:	learn: 0.2435453	total: 3.21s	remaining: 6.13s
344:	learn: 0.2434113	total: 3.22s	remaining: 6.12s
345:	learn: 0.2432653	total: 3.23s	remaining: 6.11s
346:	learn: 0.2431352	total: 3.24s	remaining: 6.1s
347:	learn: 0.2428993	total: 3.25s	remaining: 6.09s
348:	learn: 0.2427887	total: 3.26s	remaining: 6.08s
349:	learn: 0.2426357	total: 3.27s	remaining: 6.07s
350:	learn: 0.2425497	total: 3.28s	remaining: 6.06s
351:	learn: 0.2424273	total: 3.29s	remaining: 6.05s
352:	learn: 0.2422452	total: 3.29s	remaining: 6.04s
353:	learn: 0.2420839	total: 3.3s	remaining: 6.03s
354:	learn: 0.2419085	total: 3.31s	remaining: 6.01s
355:	learn: 0.2418314	total: 3.32s	remaining: 6s
356:	learn: 0.2416975	total: 3.33s	remaining: 5.99s
357:	learn: 0.2414923	total: 3.34s	remaining: 5.98s
358:	learn: 0.2413327	total: 3.35s	remaining: 5.97s
359:	learn: 0.2411316	total: 3.35s	remaining: 5.96s
360:	learn: 0.2410124	total: 3.36s	remaining: 5.95s
361:	learn: 0.2409290	total: 3.37s	remaining: 5.94s
362:	learn: 0.2408180	total: 3.38s	remaining: 5.93s
363:	learn: 0.2406604	total: 3.39s	remaining: 5.92s
364:	learn: 0.2405462	total: 3.4s	remaining: 5.91s
365:	learn: 0.2403497	total: 3.41s	remaining: 5.9s
366:	learn: 0.2402186	total: 3.42s	remaining: 5.89s
367:	learn: 0.2400975	total: 3.42s	remaining: 5.88s
368:	learn: 0.2400155	total: 3.43s	remaining: 5.87s
369:	learn: 0.2399089	total: 3.44s	remaining: 5.86s
370:	learn: 0.2397497	total: 3.45s	remaining: 5.85s
371:	learn: 0.2395609	total: 3.46s	remaining: 5.84s
372:	learn: 0.2394712	total: 3.47s	remaining: 5.83s
373:	learn: 0.2392795	total: 3.48s	remaining: 5.83s
374:	learn: 0.2391077	total: 3.49s	remaining: 5.82s
375:	learn: 0.2389591	total: 3.5s	remaining: 5.8s
376:	learn: 0.2387975	total: 3.51s	remaining: 5.79s
377:	learn: 0.2386475	total: 3.51s	remaining: 5.78s
378:	learn: 0.2384583	total: 3.52s	remaining: 5.77s
379:	learn: 0.2383246	total: 3.53s	remaining: 5.76s
380:	learn: 0.2382125	total: 3.54s	remaining: 5.75s
381:	learn: 0.2380795	total: 3.55s	remaining: 5.74s
382:	learn: 0.2379723	total: 3.56s	remaining: 5.73s
383:	learn: 0.2378424	total: 3.57s	remaining: 5.72s
384:	learn: 0.2376807	total: 3.58s	remaining: 5.71s
385:	learn: 0.2375289	total: 3.58s	remaining: 5.7s
386:	learn: 0.2373615	total: 3.59s	remaining: 5.69s
387:	learn: 0.2372254	total: 3.6s	remaining: 5.68s
388:	learn: 0.2370385	total: 3.61s	remaining: 5.67s
389:	learn: 0.2369145	total: 3.62s	remaining: 5.66s
390:	learn: 0.2367452	total: 3.63s	remaining: 5.65s
391:	learn: 0.2366130	total: 3.64s	remaining: 5.64s
392:	learn: 0.2364555	total: 3.65s	remaining: 5.63s
393:	learn: 0.2363208	total: 3.66s	remaining: 5.62s
394:	learn: 0.2361514	total: 3.67s	remaining: 5.62s
395:	learn: 0.2360220	total: 3.67s	remaining: 5.61s
396:	learn: 0.2358756	total: 3.68s	remaining: 5.6s
397:	learn: 0.2357259	total: 3.69s	remaining: 5.59s
398:	learn: 0.2355434	total: 3.7s	remaining: 5.58s
399:	learn: 0.2353784	total: 3.71s	remaining: 5.57s
400:	learn: 0.2352491	total: 3.72s	remaining: 5.56s
401:	learn: 0.2351260	total: 3.73s	remaining: 5.55s
402:	learn: 0.2349609	total: 3.74s	remaining: 5.54s
403:	learn: 0.2347994	total: 3.75s	remaining: 5.53s
404:	learn: 0.2346916	total: 3.75s	remaining: 5.52s
405:	learn: 0.2345687	total: 3.76s	remaining: 5.5s
406:	learn: 0.2344303	total: 3.77s	remaining: 5.5s
407:	learn: 0.2343390	total: 3.78s	remaining: 5.48s
408:	learn: 0.2342180	total: 3.79s	remaining: 5.47s
409:	learn: 0.2341418	total: 3.8s	remaining: 5.46s
410:	learn: 0.2340184	total: 3.81s	remaining: 5.45s
411:	learn: 0.2338744	total: 3.81s	remaining: 5.45s
412:	learn: 0.2337061	total: 3.82s	remaining: 5.44s
413:	learn: 0.2335720	total: 3.83s	remaining: 5.43s
414:	learn: 0.2334729	total: 3.84s	remaining: 5.42s
415:	learn: 0.2333335	total: 3.85s	remaining: 5.41s
416:	learn: 0.2332302	total: 3.86s	remaining: 5.4s
417:	learn: 0.2330398	total: 3.87s	remaining: 5.38s
418:	learn: 0.2329418	total: 3.88s	remaining: 5.38s
419:	learn: 0.2327961	total: 3.88s	remaining: 5.37s
420:	learn: 0.2326611	total: 3.89s	remaining: 5.35s
421:	learn: 0.2325602	total: 3.9s	remaining: 5.34s
422:	learn: 0.2324539	total: 3.91s	remaining: 5.33s
423:	learn: 0.2322812	total: 3.92s	remaining: 5.32s
424:	learn: 0.2321869	total: 3.93s	remaining: 5.31s
425:	learn: 0.2320569	total: 3.94s	remaining: 5.3s
426:	learn: 0.2319463	total: 3.95s	remaining: 5.29s
427:	learn: 0.2318125	total: 3.95s	remaining: 5.29s
428:	learn: 0.2317104	total: 3.96s	remaining: 5.28s
429:	learn: 0.2315924	total: 3.97s	remaining: 5.26s
430:	learn: 0.2313938	total: 3.98s	remaining: 5.25s
431:	learn: 0.2312348	total: 4s	remaining: 5.26s
432:	learn: 0.2311276	total: 4.01s	remaining: 5.25s
433:	learn: 0.2309550	total: 4.02s	remaining: 5.24s
434:	learn: 0.2308841	total: 4.03s	remaining: 5.23s
435:	learn: 0.2307610	total: 4.04s	remaining: 5.22s
436:	learn: 0.2305674	total: 4.04s	remaining: 5.21s
437:	learn: 0.2305346	total: 4.05s	remaining: 5.2s
438:	learn: 0.2304477	total: 4.06s	remaining: 5.19s
439:	learn: 0.2302471	total: 4.07s	remaining: 5.18s
440:	learn: 0.2301297	total: 4.08s	remaining: 5.17s
441:	learn: 0.2299658	total: 4.09s	remaining: 5.16s
442:	learn: 0.2298229	total: 4.09s	remaining: 5.15s
443:	learn: 0.2296981	total: 4.1s	remaining: 5.14s
444:	learn: 0.2295897	total: 4.11s	remaining: 5.13s
445:	learn: 0.2294617	total: 4.12s	remaining: 5.12s
446:	learn: 0.2292636	total: 4.13s	remaining: 5.11s
447:	learn: 0.2290406	total: 4.14s	remaining: 5.1s
448:	learn: 0.2289086	total: 4.15s	remaining: 5.09s
449:	learn: 0.2287827	total: 4.16s	remaining: 5.08s
450:	learn: 0.2286557	total: 4.16s	remaining: 5.07s
451:	learn: 0.2285447	total: 4.17s	remaining: 5.06s
452:	learn: 0.2284336	total: 4.18s	remaining: 5.05s
453:	learn: 0.2283532	total: 4.19s	remaining: 5.04s
454:	learn: 0.2282081	total: 4.2s	remaining: 5.03s
455:	learn: 0.2281120	total: 4.21s	remaining: 5.02s
456:	learn: 0.2280325	total: 4.22s	remaining: 5.01s
457:	learn: 0.2279067	total: 4.23s	remaining: 5s
458:	learn: 0.2277507	total: 4.24s	remaining: 5s
459:	learn: 0.2276237	total: 4.25s	remaining: 4.99s
460:	learn: 0.2274905	total: 4.26s	remaining: 4.99s
461:	learn: 0.2274061	total: 4.27s	remaining: 4.98s
462:	learn: 0.2272695	total: 4.28s	remaining: 4.97s
463:	learn: 0.2271701	total: 4.29s	remaining: 4.96s
464:	learn: 0.2270513	total: 4.3s	remaining: 4.95s
465:	learn: 0.2269151	total: 4.31s	remaining: 4.94s
466:	learn: 0.2267380	total: 4.33s	remaining: 4.94s
467:	learn: 0.2265848	total: 4.34s	remaining: 4.93s
468:	learn: 0.2264528	total: 4.34s	remaining: 4.92s
469:	learn: 0.2263277	total: 4.35s	remaining: 4.91s
470:	learn: 0.2261721	total: 4.36s	remaining: 4.9s
471:	learn: 0.2260207	total: 4.37s	remaining: 4.89s
472:	learn: 0.2259198	total: 4.38s	remaining: 4.88s
473:	learn: 0.2257243	total: 4.39s	remaining: 4.87s
474:	learn: 0.2255546	total: 4.4s	remaining: 4.86s
475:	learn: 0.2254678	total: 4.41s	remaining: 4.85s
476:	learn: 0.2253051	total: 4.42s	remaining: 4.84s
477:	learn: 0.2251335	total: 4.43s	remaining: 4.83s
478:	learn: 0.2249363	total: 4.44s	remaining: 4.82s
479:	learn: 0.2248331	total: 4.45s	remaining: 4.82s
480:	learn: 0.2247060	total: 4.46s	remaining: 4.81s
481:	learn: 0.2246219	total: 4.46s	remaining: 4.8s
482:	learn: 0.2245236	total: 4.47s	remaining: 4.79s
483:	learn: 0.2243928	total: 4.48s	remaining: 4.78s
484:	learn: 0.2243090	total: 4.49s	remaining: 4.77s
485:	learn: 0.2242068	total: 4.5s	remaining: 4.76s
486:	learn: 0.2241071	total: 4.51s	remaining: 4.75s
487:	learn: 0.2239461	total: 4.52s	remaining: 4.74s
488:	learn: 0.2238371	total: 4.53s	remaining: 4.73s
489:	learn: 0.2237369	total: 4.54s	remaining: 4.72s
490:	learn: 0.2236514	total: 4.54s	remaining: 4.71s
491:	learn: 0.2235400	total: 4.55s	remaining: 4.7s
492:	learn: 0.2234295	total: 4.56s	remaining: 4.69s
493:	learn: 0.2233072	total: 4.57s	remaining: 4.68s
494:	learn: 0.2232001	total: 4.58s	remaining: 4.67s
495:	learn: 0.2230219	total: 4.59s	remaining: 4.66s
496:	learn: 0.2229117	total: 4.59s	remaining: 4.65s
497:	learn: 0.2227602	total: 4.6s	remaining: 4.64s
498:	learn: 0.2226179	total: 4.61s	remaining: 4.63s
499:	learn: 0.2225213	total: 4.62s	remaining: 4.62s
500:	learn: 0.2224048	total: 4.63s	remaining: 4.61s
501:	learn: 0.2222955	total: 4.64s	remaining: 4.6s
502:	learn: 0.2221325	total: 4.65s	remaining: 4.59s
503:	learn: 0.2220017	total: 4.66s	remaining: 4.58s
504:	learn: 0.2218458	total: 4.67s	remaining: 4.58s
505:	learn: 0.2217328	total: 4.68s	remaining: 4.57s
506:	learn: 0.2216059	total: 4.69s	remaining: 4.56s
507:	learn: 0.2214840	total: 4.7s	remaining: 4.55s
508:	learn: 0.2213401	total: 4.7s	remaining: 4.54s
509:	learn: 0.2212241	total: 4.71s	remaining: 4.53s
510:	learn: 0.2210771	total: 4.72s	remaining: 4.52s
511:	learn: 0.2209674	total: 4.73s	remaining: 4.51s
512:	learn: 0.2208301	total: 4.74s	remaining: 4.5s
513:	learn: 0.2207277	total: 4.75s	remaining: 4.49s
514:	learn: 0.2206409	total: 4.76s	remaining: 4.48s
515:	learn: 0.2205040	total: 4.77s	remaining: 4.47s
516:	learn: 0.2203624	total: 4.78s	remaining: 4.46s
517:	learn: 0.2202521	total: 4.79s	remaining: 4.45s
518:	learn: 0.2201437	total: 4.79s	remaining: 4.44s
519:	learn: 0.2200607	total: 4.8s	remaining: 4.43s
520:	learn: 0.2198786	total: 4.81s	remaining: 4.42s
521:	learn: 0.2197322	total: 4.82s	remaining: 4.42s
522:	learn: 0.2195702	total: 4.83s	remaining: 4.41s
523:	learn: 0.2194651	total: 4.84s	remaining: 4.4s
524:	learn: 0.2193371	total: 4.85s	remaining: 4.39s
525:	learn: 0.2192634	total: 4.86s	remaining: 4.38s
526:	learn: 0.2191587	total: 4.87s	remaining: 4.37s
527:	learn: 0.2190038	total: 4.87s	remaining: 4.36s
528:	learn: 0.2188734	total: 4.88s	remaining: 4.35s
529:	learn: 0.2187285	total: 4.89s	remaining: 4.34s
530:	learn: 0.2186335	total: 4.9s	remaining: 4.33s
531:	learn: 0.2184712	total: 4.91s	remaining: 4.32s
532:	learn: 0.2183729	total: 4.92s	remaining: 4.31s
533:	learn: 0.2182708	total: 4.92s	remaining: 4.3s
534:	learn: 0.2181324	total: 4.93s	remaining: 4.29s
535:	learn: 0.2179661	total: 4.94s	remaining: 4.28s
536:	learn: 0.2178662	total: 4.95s	remaining: 4.27s
537:	learn: 0.2177851	total: 4.96s	remaining: 4.26s
538:	learn: 0.2176486	total: 4.97s	remaining: 4.25s
539:	learn: 0.2175338	total: 4.98s	remaining: 4.24s
540:	learn: 0.2173977	total: 4.98s	remaining: 4.23s
541:	learn: 0.2172046	total: 4.99s	remaining: 4.22s
542:	learn: 0.2170623	total: 5.01s	remaining: 4.22s
543:	learn: 0.2169273	total: 5.03s	remaining: 4.21s
544:	learn: 0.2168617	total: 5.03s	remaining: 4.2s
545:	learn: 0.2167891	total: 5.04s	remaining: 4.19s
546:	learn: 0.2166816	total: 5.05s	remaining: 4.18s
547:	learn: 0.2165086	total: 5.06s	remaining: 4.17s
548:	learn: 0.2164122	total: 5.07s	remaining: 4.16s
549:	learn: 0.2162648	total: 5.08s	remaining: 4.15s
550:	learn: 0.2161712	total: 5.08s	remaining: 4.14s
551:	learn: 0.2160162	total: 5.09s	remaining: 4.13s
552:	learn: 0.2158395	total: 5.1s	remaining: 4.12s
553:	learn: 0.2156999	total: 5.11s	remaining: 4.11s
554:	learn: 0.2155332	total: 5.12s	remaining: 4.1s
555:	learn: 0.2153832	total: 5.13s	remaining: 4.09s
556:	learn: 0.2152476	total: 5.14s	remaining: 4.08s
557:	learn: 0.2151527	total: 5.14s	remaining: 4.08s
558:	learn: 0.2150890	total: 5.15s	remaining: 4.07s
559:	learn: 0.2149646	total: 5.16s	remaining: 4.06s
560:	learn: 0.2148232	total: 5.17s	remaining: 4.05s
561:	learn: 0.2147155	total: 5.18s	remaining: 4.04s
562:	learn: 0.2146310	total: 5.19s	remaining: 4.03s
563:	learn: 0.2144922	total: 5.2s	remaining: 4.02s
564:	learn: 0.2143257	total: 5.21s	remaining: 4.01s
565:	learn: 0.2141994	total: 5.22s	remaining: 4s
566:	learn: 0.2140495	total: 5.22s	remaining: 3.99s
567:	learn: 0.2139268	total: 5.23s	remaining: 3.98s
568:	learn: 0.2138160	total: 5.24s	remaining: 3.97s
569:	learn: 0.2136945	total: 5.25s	remaining: 3.96s
570:	learn: 0.2136259	total: 5.26s	remaining: 3.95s
571:	learn: 0.2135370	total: 5.27s	remaining: 3.94s
572:	learn: 0.2134327	total: 5.28s	remaining: 3.93s
573:	learn: 0.2133203	total: 5.29s	remaining: 3.92s
574:	learn: 0.2132686	total: 5.29s	remaining: 3.91s
575:	learn: 0.2131827	total: 5.3s	remaining: 3.9s
576:	learn: 0.2130532	total: 5.31s	remaining: 3.89s
577:	learn: 0.2129337	total: 5.32s	remaining: 3.88s
578:	learn: 0.2127536	total: 5.33s	remaining: 3.87s
579:	learn: 0.2126113	total: 5.34s	remaining: 3.86s
580:	learn: 0.2125353	total: 5.34s	remaining: 3.85s
581:	learn: 0.2124490	total: 5.35s	remaining: 3.84s
582:	learn: 0.2123828	total: 5.36s	remaining: 3.83s
583:	learn: 0.2122520	total: 5.37s	remaining: 3.83s
584:	learn: 0.2120871	total: 5.38s	remaining: 3.82s
585:	learn: 0.2119584	total: 5.39s	remaining: 3.81s
586:	learn: 0.2118053	total: 5.4s	remaining: 3.8s
587:	learn: 0.2117524	total: 5.41s	remaining: 3.79s
588:	learn: 0.2116625	total: 5.42s	remaining: 3.78s
589:	learn: 0.2115743	total: 5.42s	remaining: 3.77s
590:	learn: 0.2113882	total: 5.43s	remaining: 3.76s
591:	learn: 0.2112835	total: 5.44s	remaining: 3.75s
592:	learn: 0.2111451	total: 5.45s	remaining: 3.74s
593:	learn: 0.2110287	total: 5.46s	remaining: 3.73s
594:	learn: 0.2109218	total: 5.47s	remaining: 3.72s
595:	learn: 0.2108180	total: 5.48s	remaining: 3.71s
596:	learn: 0.2107286	total: 5.49s	remaining: 3.71s
597:	learn: 0.2104909	total: 5.5s	remaining: 3.7s
598:	learn: 0.2104028	total: 5.51s	remaining: 3.69s
599:	learn: 0.2103004	total: 5.52s	remaining: 3.68s
600:	learn: 0.2102437	total: 5.53s	remaining: 3.67s
601:	learn: 0.2101214	total: 5.54s	remaining: 3.66s
602:	learn: 0.2100038	total: 5.55s	remaining: 3.65s
603:	learn: 0.2098937	total: 5.56s	remaining: 3.64s
604:	learn: 0.2097939	total: 5.57s	remaining: 3.63s
605:	learn: 0.2097039	total: 5.58s	remaining: 3.63s
606:	learn: 0.2095783	total: 5.59s	remaining: 3.62s
607:	learn: 0.2094771	total: 5.6s	remaining: 3.61s
608:	learn: 0.2093728	total: 5.61s	remaining: 3.6s
609:	learn: 0.2092306	total: 5.62s	remaining: 3.59s
610:	learn: 0.2091699	total: 5.63s	remaining: 3.58s
611:	learn: 0.2090960	total: 5.63s	remaining: 3.57s
612:	learn: 0.2089333	total: 5.64s	remaining: 3.56s
613:	learn: 0.2087970	total: 5.65s	remaining: 3.55s
614:	learn: 0.2086565	total: 5.66s	remaining: 3.54s
615:	learn: 0.2085229	total: 5.67s	remaining: 3.53s
616:	learn: 0.2084539	total: 5.68s	remaining: 3.52s
617:	learn: 0.2083963	total: 5.69s	remaining: 3.52s
618:	learn: 0.2083443	total: 5.7s	remaining: 3.51s
619:	learn: 0.2082679	total: 5.71s	remaining: 3.5s
620:	learn: 0.2082152	total: 5.71s	remaining: 3.49s
621:	learn: 0.2080823	total: 5.73s	remaining: 3.48s
622:	learn: 0.2079806	total: 5.74s	remaining: 3.47s
623:	learn: 0.2078694	total: 5.75s	remaining: 3.46s
624:	learn: 0.2077670	total: 5.76s	remaining: 3.45s
625:	learn: 0.2076134	total: 5.77s	remaining: 3.44s
626:	learn: 0.2075270	total: 5.78s	remaining: 3.44s
627:	learn: 0.2074015	total: 5.79s	remaining: 3.43s
628:	learn: 0.2072918	total: 5.79s	remaining: 3.42s
629:	learn: 0.2071480	total: 5.8s	remaining: 3.41s
630:	learn: 0.2070629	total: 5.81s	remaining: 3.4s
631:	learn: 0.2068901	total: 5.82s	remaining: 3.39s
632:	learn: 0.2068071	total: 5.83s	remaining: 3.38s
633:	learn: 0.2066510	total: 5.84s	remaining: 3.37s
634:	learn: 0.2065085	total: 5.87s	remaining: 3.37s
635:	learn: 0.2064441	total: 5.88s	remaining: 3.36s
636:	learn: 0.2063597	total: 5.89s	remaining: 3.35s
637:	learn: 0.2062872	total: 5.9s	remaining: 3.35s
638:	learn: 0.2061199	total: 5.91s	remaining: 3.34s
639:	learn: 0.2060264	total: 5.91s	remaining: 3.33s
640:	learn: 0.2059275	total: 5.92s	remaining: 3.32s
641:	learn: 0.2058374	total: 5.93s	remaining: 3.31s
642:	learn: 0.2056859	total: 5.94s	remaining: 3.3s
643:	learn: 0.2056582	total: 5.95s	remaining: 3.29s
644:	learn: 0.2055508	total: 5.96s	remaining: 3.28s
645:	learn: 0.2054240	total: 5.97s	remaining: 3.27s
646:	learn: 0.2052568	total: 5.98s	remaining: 3.26s
647:	learn: 0.2051464	total: 5.99s	remaining: 3.25s
648:	learn: 0.2050658	total: 6s	remaining: 3.25s
649:	learn: 0.2049983	total: 6.03s	remaining: 3.25s
650:	learn: 0.2048593	total: 6.04s	remaining: 3.24s
651:	learn: 0.2047436	total: 6.05s	remaining: 3.23s
652:	learn: 0.2046263	total: 6.06s	remaining: 3.22s
653:	learn: 0.2045206	total: 6.07s	remaining: 3.21s
654:	learn: 0.2044644	total: 6.08s	remaining: 3.2s
655:	learn: 0.2043938	total: 6.08s	remaining: 3.19s
656:	learn: 0.2042117	total: 6.09s	remaining: 3.18s
657:	learn: 0.2040681	total: 6.1s	remaining: 3.17s
658:	learn: 0.2039429	total: 6.11s	remaining: 3.16s
659:	learn: 0.2038139	total: 6.12s	remaining: 3.15s
660:	learn: 0.2037446	total: 6.13s	remaining: 3.15s
661:	learn: 0.2036243	total: 6.14s	remaining: 3.14s
662:	learn: 0.2034967	total: 6.15s	remaining: 3.13s
663:	learn: 0.2034213	total: 6.16s	remaining: 3.12s
664:	learn: 0.2033351	total: 6.17s	remaining: 3.11s
665:	learn: 0.2032464	total: 6.18s	remaining: 3.1s
666:	learn: 0.2031178	total: 6.19s	remaining: 3.09s
667:	learn: 0.2030897	total: 6.2s	remaining: 3.08s
668:	learn: 0.2030044	total: 6.21s	remaining: 3.07s
669:	learn: 0.2028892	total: 6.22s	remaining: 3.06s
670:	learn: 0.2027634	total: 6.23s	remaining: 3.05s
671:	learn: 0.2026963	total: 6.24s	remaining: 3.04s
672:	learn: 0.2025999	total: 6.25s	remaining: 3.04s
673:	learn: 0.2024389	total: 6.26s	remaining: 3.03s
674:	learn: 0.2023066	total: 6.27s	remaining: 3.02s
675:	learn: 0.2022085	total: 6.28s	remaining: 3.01s
676:	learn: 0.2021352	total: 6.29s	remaining: 3s
677:	learn: 0.2020618	total: 6.29s	remaining: 2.99s
678:	learn: 0.2019450	total: 6.3s	remaining: 2.98s
679:	learn: 0.2018868	total: 6.31s	remaining: 2.97s
680:	learn: 0.2018401	total: 6.32s	remaining: 2.96s
681:	learn: 0.2017235	total: 6.33s	remaining: 2.95s
682:	learn: 0.2016072	total: 6.34s	remaining: 2.94s
683:	learn: 0.2015014	total: 6.35s	remaining: 2.94s
684:	learn: 0.2014332	total: 6.36s	remaining: 2.93s
685:	learn: 0.2013360	total: 6.37s	remaining: 2.92s
686:	learn: 0.2012819	total: 6.38s	remaining: 2.91s
687:	learn: 0.2011922	total: 6.39s	remaining: 2.9s
688:	learn: 0.2011158	total: 6.4s	remaining: 2.89s
689:	learn: 0.2009604	total: 6.41s	remaining: 2.88s
690:	learn: 0.2008475	total: 6.42s	remaining: 2.87s
691:	learn: 0.2007763	total: 6.43s	remaining: 2.86s
692:	learn: 0.2007123	total: 6.44s	remaining: 2.85s
693:	learn: 0.2005868	total: 6.45s	remaining: 2.84s
694:	learn: 0.2004046	total: 6.46s	remaining: 2.83s
695:	learn: 0.2002514	total: 6.47s	remaining: 2.83s
696:	learn: 0.2001676	total: 6.48s	remaining: 2.82s
697:	learn: 0.2001014	total: 6.49s	remaining: 2.81s
698:	learn: 0.2000354	total: 6.5s	remaining: 2.8s
699:	learn: 0.1999966	total: 6.51s	remaining: 2.79s
700:	learn: 0.1999178	total: 6.52s	remaining: 2.78s
701:	learn: 0.1998393	total: 6.53s	remaining: 2.77s
702:	learn: 0.1997021	total: 6.54s	remaining: 2.76s
703:	learn: 0.1996132	total: 6.55s	remaining: 2.75s
704:	learn: 0.1995008	total: 6.56s	remaining: 2.75s
705:	learn: 0.1994306	total: 6.57s	remaining: 2.74s
706:	learn: 0.1993650	total: 6.58s	remaining: 2.73s
707:	learn: 0.1992605	total: 6.59s	remaining: 2.72s
708:	learn: 0.1991203	total: 6.6s	remaining: 2.71s
709:	learn: 0.1990744	total: 6.61s	remaining: 2.7s
710:	learn: 0.1989432	total: 6.62s	remaining: 2.69s
711:	learn: 0.1987643	total: 6.63s	remaining: 2.68s
712:	learn: 0.1986177	total: 6.64s	remaining: 2.67s
713:	learn: 0.1985549	total: 6.65s	remaining: 2.66s
714:	learn: 0.1984282	total: 6.66s	remaining: 2.65s
715:	learn: 0.1983796	total: 6.67s	remaining: 2.64s
716:	learn: 0.1983275	total: 6.68s	remaining: 2.64s
717:	learn: 0.1982238	total: 6.7s	remaining: 2.63s
718:	learn: 0.1981291	total: 6.7s	remaining: 2.62s
719:	learn: 0.1980462	total: 6.71s	remaining: 2.61s
720:	learn: 0.1979671	total: 6.72s	remaining: 2.6s
721:	learn: 0.1978990	total: 6.73s	remaining: 2.59s
722:	learn: 0.1978289	total: 6.74s	remaining: 2.58s
723:	learn: 0.1977390	total: 6.75s	remaining: 2.57s
724:	learn: 0.1976629	total: 6.76s	remaining: 2.56s
725:	learn: 0.1975550	total: 6.77s	remaining: 2.56s
726:	learn: 0.1974396	total: 6.78s	remaining: 2.55s
727:	learn: 0.1973852	total: 6.79s	remaining: 2.54s
728:	learn: 0.1972859	total: 6.8s	remaining: 2.53s
729:	learn: 0.1972594	total: 6.81s	remaining: 2.52s
730:	learn: 0.1971583	total: 6.82s	remaining: 2.51s
731:	learn: 0.1970390	total: 6.83s	remaining: 2.5s
732:	learn: 0.1969296	total: 6.84s	remaining: 2.49s
733:	learn: 0.1968148	total: 6.85s	remaining: 2.48s
734:	learn: 0.1967473	total: 6.86s	remaining: 2.47s
735:	learn: 0.1966606	total: 6.87s	remaining: 2.46s
736:	learn: 0.1965283	total: 6.88s	remaining: 2.45s
737:	learn: 0.1963679	total: 6.88s	remaining: 2.44s
738:	learn: 0.1962207	total: 6.89s	remaining: 2.44s
739:	learn: 0.1960526	total: 6.91s	remaining: 2.43s
740:	learn: 0.1959885	total: 6.92s	remaining: 2.42s
741:	learn: 0.1959484	total: 6.92s	remaining: 2.41s
742:	learn: 0.1958397	total: 6.93s	remaining: 2.4s
743:	learn: 0.1957125	total: 6.94s	remaining: 2.39s
744:	learn: 0.1956444	total: 6.95s	remaining: 2.38s
745:	learn: 0.1955673	total: 6.96s	remaining: 2.37s
746:	learn: 0.1955154	total: 6.97s	remaining: 2.36s
747:	learn: 0.1953868	total: 6.98s	remaining: 2.35s
748:	learn: 0.1953089	total: 6.99s	remaining: 2.34s
749:	learn: 0.1951873	total: 7s	remaining: 2.33s
750:	learn: 0.1951182	total: 7.01s	remaining: 2.33s
751:	learn: 0.1950463	total: 7.02s	remaining: 2.32s
752:	learn: 0.1949470	total: 7.05s	remaining: 2.31s
753:	learn: 0.1948002	total: 7.06s	remaining: 2.3s
754:	learn: 0.1946782	total: 7.07s	remaining: 2.29s
755:	learn: 0.1946093	total: 7.08s	remaining: 2.29s
756:	learn: 0.1944906	total: 7.09s	remaining: 2.28s
757:	learn: 0.1943957	total: 7.1s	remaining: 2.27s
758:	learn: 0.1942928	total: 7.11s	remaining: 2.26s
759:	learn: 0.1942170	total: 7.12s	remaining: 2.25s
760:	learn: 0.1941186	total: 7.13s	remaining: 2.24s
761:	learn: 0.1940506	total: 7.14s	remaining: 2.23s
762:	learn: 0.1939629	total: 7.15s	remaining: 2.22s
763:	learn: 0.1938914	total: 7.16s	remaining: 2.21s
764:	learn: 0.1938302	total: 7.17s	remaining: 2.2s
765:	learn: 0.1936460	total: 7.18s	remaining: 2.19s
766:	learn: 0.1935484	total: 7.19s	remaining: 2.18s
767:	learn: 0.1934662	total: 7.2s	remaining: 2.17s
768:	learn: 0.1933517	total: 7.21s	remaining: 2.16s
769:	learn: 0.1932679	total: 7.21s	remaining: 2.15s
770:	learn: 0.1932146	total: 7.22s	remaining: 2.15s
771:	learn: 0.1931580	total: 7.23s	remaining: 2.14s
772:	learn: 0.1930608	total: 7.24s	remaining: 2.13s
773:	learn: 0.1929738	total: 7.25s	remaining: 2.12s
774:	learn: 0.1928951	total: 7.26s	remaining: 2.11s
775:	learn: 0.1928149	total: 7.27s	remaining: 2.1s
776:	learn: 0.1927089	total: 7.29s	remaining: 2.09s
777:	learn: 0.1925211	total: 7.3s	remaining: 2.08s
778:	learn: 0.1923738	total: 7.31s	remaining: 2.07s
779:	learn: 0.1923381	total: 7.32s	remaining: 2.06s
780:	learn: 0.1921897	total: 7.33s	remaining: 2.06s
781:	learn: 0.1920827	total: 7.34s	remaining: 2.05s
782:	learn: 0.1919364	total: 7.35s	remaining: 2.04s
783:	learn: 0.1917910	total: 7.36s	remaining: 2.03s
784:	learn: 0.1916897	total: 7.37s	remaining: 2.02s
785:	learn: 0.1916133	total: 7.38s	remaining: 2.01s
786:	learn: 0.1914993	total: 7.39s	remaining: 2s
787:	learn: 0.1913764	total: 7.4s	remaining: 1.99s
788:	learn: 0.1912800	total: 7.41s	remaining: 1.98s
789:	learn: 0.1911654	total: 7.42s	remaining: 1.97s
790:	learn: 0.1911265	total: 7.43s	remaining: 1.96s
791:	learn: 0.1910414	total: 7.44s	remaining: 1.95s
792:	learn: 0.1909511	total: 7.45s	remaining: 1.94s
793:	learn: 0.1908665	total: 7.46s	remaining: 1.93s
794:	learn: 0.1907065	total: 7.47s	remaining: 1.93s
795:	learn: 0.1906451	total: 7.48s	remaining: 1.92s
796:	learn: 0.1905599	total: 7.49s	remaining: 1.91s
797:	learn: 0.1904331	total: 7.5s	remaining: 1.9s
798:	learn: 0.1903862	total: 7.51s	remaining: 1.89s
799:	learn: 0.1902897	total: 7.52s	remaining: 1.88s
800:	learn: 0.1902358	total: 7.53s	remaining: 1.87s
801:	learn: 0.1901234	total: 7.53s	remaining: 1.86s
802:	learn: 0.1900622	total: 7.54s	remaining: 1.85s
803:	learn: 0.1899813	total: 7.55s	remaining: 1.84s
804:	learn: 0.1898638	total: 7.56s	remaining: 1.83s
805:	learn: 0.1897952	total: 7.57s	remaining: 1.82s
806:	learn: 0.1897101	total: 7.58s	remaining: 1.81s
807:	learn: 0.1896124	total: 7.59s	remaining: 1.8s
808:	learn: 0.1895211	total: 7.59s	remaining: 1.79s
809:	learn: 0.1894348	total: 7.6s	remaining: 1.78s
810:	learn: 0.1893045	total: 7.61s	remaining: 1.77s
811:	learn: 0.1891806	total: 7.62s	remaining: 1.76s
812:	learn: 0.1890739	total: 7.63s	remaining: 1.75s
813:	learn: 0.1889652	total: 7.64s	remaining: 1.75s
814:	learn: 0.1888606	total: 7.65s	remaining: 1.74s
815:	learn: 0.1887620	total: 7.66s	remaining: 1.73s
816:	learn: 0.1887163	total: 7.67s	remaining: 1.72s
817:	learn: 0.1886679	total: 7.68s	remaining: 1.71s
818:	learn: 0.1885343	total: 7.68s	remaining: 1.7s
819:	learn: 0.1884173	total: 7.69s	remaining: 1.69s
820:	learn: 0.1883315	total: 7.7s	remaining: 1.68s
821:	learn: 0.1882177	total: 7.71s	remaining: 1.67s
822:	learn: 0.1880899	total: 7.72s	remaining: 1.66s
823:	learn: 0.1879901	total: 7.73s	remaining: 1.65s
824:	learn: 0.1878798	total: 7.74s	remaining: 1.64s
825:	learn: 0.1877830	total: 7.75s	remaining: 1.63s
826:	learn: 0.1876515	total: 7.75s	remaining: 1.62s
827:	learn: 0.1875839	total: 7.76s	remaining: 1.61s
828:	learn: 0.1874984	total: 7.77s	remaining: 1.6s
829:	learn: 0.1873354	total: 7.78s	remaining: 1.59s
830:	learn: 0.1871765	total: 7.79s	remaining: 1.58s
831:	learn: 0.1870846	total: 7.8s	remaining: 1.57s
832:	learn: 0.1869671	total: 7.81s	remaining: 1.56s
833:	learn: 0.1868693	total: 7.82s	remaining: 1.55s
834:	learn: 0.1867897	total: 7.82s	remaining: 1.55s
835:	learn: 0.1867156	total: 7.83s	remaining: 1.54s
836:	learn: 0.1866729	total: 7.84s	remaining: 1.53s
837:	learn: 0.1865581	total: 7.85s	remaining: 1.52s
838:	learn: 0.1864680	total: 7.86s	remaining: 1.51s
839:	learn: 0.1863957	total: 7.87s	remaining: 1.5s
840:	learn: 0.1863220	total: 7.88s	remaining: 1.49s
841:	learn: 0.1862263	total: 7.89s	remaining: 1.48s
842:	learn: 0.1861147	total: 7.89s	remaining: 1.47s
843:	learn: 0.1860306	total: 7.9s	remaining: 1.46s
844:	learn: 0.1859270	total: 7.91s	remaining: 1.45s
845:	learn: 0.1857666	total: 7.92s	remaining: 1.44s
846:	learn: 0.1856550	total: 7.93s	remaining: 1.43s
847:	learn: 0.1855864	total: 7.94s	remaining: 1.42s
848:	learn: 0.1854999	total: 7.95s	remaining: 1.41s
849:	learn: 0.1853941	total: 7.95s	remaining: 1.4s
850:	learn: 0.1853014	total: 7.96s	remaining: 1.39s
851:	learn: 0.1851548	total: 7.97s	remaining: 1.38s
852:	learn: 0.1850907	total: 7.98s	remaining: 1.38s
853:	learn: 0.1850042	total: 7.99s	remaining: 1.36s
854:	learn: 0.1848861	total: 8s	remaining: 1.36s
855:	learn: 0.1848646	total: 8.01s	remaining: 1.35s
856:	learn: 0.1848044	total: 8.01s	remaining: 1.34s
857:	learn: 0.1847133	total: 8.02s	remaining: 1.33s
858:	learn: 0.1846143	total: 8.03s	remaining: 1.32s
859:	learn: 0.1845560	total: 8.04s	remaining: 1.31s
860:	learn: 0.1844811	total: 8.05s	remaining: 1.3s
861:	learn: 0.1843629	total: 8.09s	remaining: 1.29s
862:	learn: 0.1842635	total: 8.09s	remaining: 1.28s
863:	learn: 0.1841471	total: 8.1s	remaining: 1.27s
864:	learn: 0.1840235	total: 8.11s	remaining: 1.27s
865:	learn: 0.1839029	total: 8.12s	remaining: 1.26s
866:	learn: 0.1837678	total: 8.13s	remaining: 1.25s
867:	learn: 0.1836684	total: 8.14s	remaining: 1.24s
868:	learn: 0.1836005	total: 8.15s	remaining: 1.23s
869:	learn: 0.1835241	total: 8.16s	remaining: 1.22s
870:	learn: 0.1834242	total: 8.16s	remaining: 1.21s
871:	learn: 0.1833506	total: 8.17s	remaining: 1.2s
872:	learn: 0.1832906	total: 8.18s	remaining: 1.19s
873:	learn: 0.1831817	total: 8.19s	remaining: 1.18s
874:	learn: 0.1830536	total: 8.2s	remaining: 1.17s
875:	learn: 0.1828993	total: 8.21s	remaining: 1.16s
876:	learn: 0.1828154	total: 8.22s	remaining: 1.15s
877:	learn: 0.1827272	total: 8.23s	remaining: 1.14s
878:	learn: 0.1826231	total: 8.24s	remaining: 1.13s
879:	learn: 0.1825376	total: 8.24s	remaining: 1.12s
880:	learn: 0.1824464	total: 8.25s	remaining: 1.11s
881:	learn: 0.1823522	total: 8.26s	remaining: 1.1s
882:	learn: 0.1822651	total: 8.27s	remaining: 1.1s
883:	learn: 0.1821848	total: 8.28s	remaining: 1.09s
884:	learn: 0.1821301	total: 8.29s	remaining: 1.08s
885:	learn: 0.1820125	total: 8.3s	remaining: 1.07s
886:	learn: 0.1819399	total: 8.31s	remaining: 1.06s
887:	learn: 0.1818140	total: 8.31s	remaining: 1.05s
888:	learn: 0.1817140	total: 8.32s	remaining: 1.04s
889:	learn: 0.1815989	total: 8.33s	remaining: 1.03s
890:	learn: 0.1815156	total: 8.34s	remaining: 1.02s
891:	learn: 0.1814488	total: 8.35s	remaining: 1.01s
892:	learn: 0.1813638	total: 8.36s	remaining: 1s
893:	learn: 0.1813008	total: 8.37s	remaining: 992ms
894:	learn: 0.1812216	total: 8.38s	remaining: 983ms
895:	learn: 0.1812044	total: 8.38s	remaining: 973ms
896:	learn: 0.1811010	total: 8.39s	remaining: 964ms
897:	learn: 0.1809685	total: 8.4s	remaining: 954ms
898:	learn: 0.1809030	total: 8.41s	remaining: 945ms
899:	learn: 0.1808305	total: 8.42s	remaining: 936ms
900:	learn: 0.1807861	total: 8.43s	remaining: 926ms
901:	learn: 0.1806653	total: 8.44s	remaining: 917ms
902:	learn: 0.1805296	total: 8.45s	remaining: 907ms
903:	learn: 0.1804508	total: 8.46s	remaining: 898ms
904:	learn: 0.1803622	total: 8.46s	remaining: 888ms
905:	learn: 0.1802344	total: 8.47s	remaining: 879ms
906:	learn: 0.1801596	total: 8.48s	remaining: 870ms
907:	learn: 0.1800688	total: 8.49s	remaining: 860ms
908:	learn: 0.1799939	total: 8.5s	remaining: 851ms
909:	learn: 0.1799342	total: 8.51s	remaining: 842ms
910:	learn: 0.1798473	total: 8.52s	remaining: 832ms
911:	learn: 0.1797966	total: 8.53s	remaining: 823ms
912:	learn: 0.1796393	total: 8.54s	remaining: 814ms
913:	learn: 0.1795535	total: 8.55s	remaining: 804ms
914:	learn: 0.1794407	total: 8.56s	remaining: 795ms
915:	learn: 0.1793746	total: 8.57s	remaining: 786ms
916:	learn: 0.1793250	total: 8.57s	remaining: 776ms
917:	learn: 0.1792603	total: 8.59s	remaining: 767ms
918:	learn: 0.1791540	total: 8.59s	remaining: 758ms
919:	learn: 0.1790917	total: 8.6s	remaining: 748ms
920:	learn: 0.1790189	total: 8.61s	remaining: 739ms
921:	learn: 0.1789184	total: 8.62s	remaining: 729ms
922:	learn: 0.1788156	total: 8.63s	remaining: 720ms
923:	learn: 0.1787869	total: 8.64s	remaining: 710ms
924:	learn: 0.1786868	total: 8.65s	remaining: 701ms
925:	learn: 0.1785820	total: 8.66s	remaining: 692ms
926:	learn: 0.1784979	total: 8.66s	remaining: 682ms
927:	learn: 0.1783744	total: 8.68s	remaining: 673ms
928:	learn: 0.1783027	total: 8.68s	remaining: 664ms
929:	learn: 0.1782228	total: 8.69s	remaining: 654ms
930:	learn: 0.1781661	total: 8.7s	remaining: 645ms
931:	learn: 0.1781153	total: 8.71s	remaining: 636ms
932:	learn: 0.1780070	total: 8.72s	remaining: 626ms
933:	learn: 0.1779546	total: 8.73s	remaining: 617ms
934:	learn: 0.1778968	total: 8.74s	remaining: 607ms
935:	learn: 0.1778073	total: 8.74s	remaining: 598ms
936:	learn: 0.1777226	total: 8.75s	remaining: 589ms
937:	learn: 0.1776164	total: 8.76s	remaining: 579ms
938:	learn: 0.1775034	total: 8.77s	remaining: 570ms
939:	learn: 0.1774314	total: 8.78s	remaining: 561ms
940:	learn: 0.1773821	total: 8.79s	remaining: 551ms
941:	learn: 0.1772697	total: 8.8s	remaining: 542ms
942:	learn: 0.1772028	total: 8.81s	remaining: 532ms
943:	learn: 0.1770200	total: 8.82s	remaining: 523ms
944:	learn: 0.1769435	total: 8.82s	remaining: 514ms
945:	learn: 0.1768410	total: 8.83s	remaining: 504ms
946:	learn: 0.1767668	total: 8.84s	remaining: 495ms
947:	learn: 0.1766897	total: 8.85s	remaining: 485ms
948:	learn: 0.1765567	total: 8.86s	remaining: 476ms
949:	learn: 0.1764947	total: 8.87s	remaining: 467ms
950:	learn: 0.1764343	total: 8.88s	remaining: 457ms
951:	learn: 0.1763480	total: 8.88s	remaining: 448ms
952:	learn: 0.1762532	total: 8.89s	remaining: 439ms
953:	learn: 0.1761541	total: 8.9s	remaining: 429ms
954:	learn: 0.1760157	total: 8.91s	remaining: 420ms
955:	learn: 0.1759635	total: 8.92s	remaining: 410ms
956:	learn: 0.1758474	total: 8.93s	remaining: 401ms
957:	learn: 0.1757974	total: 8.94s	remaining: 392ms
958:	learn: 0.1757532	total: 8.94s	remaining: 382ms
959:	learn: 0.1756419	total: 8.95s	remaining: 373ms
960:	learn: 0.1756037	total: 8.96s	remaining: 364ms
961:	learn: 0.1755298	total: 8.97s	remaining: 354ms
962:	learn: 0.1754178	total: 8.98s	remaining: 345ms
963:	learn: 0.1753233	total: 8.99s	remaining: 336ms
964:	learn: 0.1752627	total: 9s	remaining: 326ms
965:	learn: 0.1751925	total: 9.01s	remaining: 317ms
966:	learn: 0.1750914	total: 9.01s	remaining: 308ms
967:	learn: 0.1750259	total: 9.02s	remaining: 298ms
968:	learn: 0.1749447	total: 9.03s	remaining: 289ms
969:	learn: 0.1748836	total: 9.04s	remaining: 280ms
970:	learn: 0.1748347	total: 9.05s	remaining: 270ms
971:	learn: 0.1747652	total: 9.06s	remaining: 261ms
972:	learn: 0.1747124	total: 9.07s	remaining: 252ms
973:	learn: 0.1746313	total: 9.1s	remaining: 243ms
974:	learn: 0.1746143	total: 9.11s	remaining: 234ms
975:	learn: 0.1745627	total: 9.12s	remaining: 224ms
976:	learn: 0.1745104	total: 9.13s	remaining: 215ms
977:	learn: 0.1744543	total: 9.13s	remaining: 205ms
978:	learn: 0.1743813	total: 9.14s	remaining: 196ms
979:	learn: 0.1743098	total: 9.15s	remaining: 187ms
980:	learn: 0.1742651	total: 9.16s	remaining: 177ms
981:	learn: 0.1742278	total: 9.17s	remaining: 168ms
982:	learn: 0.1741871	total: 9.18s	remaining: 159ms
983:	learn: 0.1740969	total: 9.19s	remaining: 149ms
984:	learn: 0.1739942	total: 9.2s	remaining: 140ms
985:	learn: 0.1739048	total: 9.2s	remaining: 131ms
986:	learn: 0.1738052	total: 9.21s	remaining: 121ms
987:	learn: 0.1737144	total: 9.22s	remaining: 112ms
988:	learn: 0.1735685	total: 9.23s	remaining: 103ms
989:	learn: 0.1734131	total: 9.24s	remaining: 93.3ms
990:	learn: 0.1733206	total: 9.25s	remaining: 84ms
991:	learn: 0.1732759	total: 9.26s	remaining: 74.6ms
992:	learn: 0.1732386	total: 9.26s	remaining: 65.3ms
993:	learn: 0.1731644	total: 9.27s	remaining: 56ms
994:	learn: 0.1730383	total: 9.28s	remaining: 46.6ms
995:	learn: 0.1729884	total: 9.29s	remaining: 37.3ms
996:	learn: 0.1729129	total: 9.3s	remaining: 28ms
997:	learn: 0.1728584	total: 9.31s	remaining: 18.6ms
998:	learn: 0.1727712	total: 9.31s	remaining: 9.32ms
999:	learn: 0.1727183	total: 9.32s	remaining: 0us

<catboost.core.CatBoostClassifier at 0x228e48eec40>

# Feature Importances : the importance of the feauture that contributed to the learning of our model sroted from most important to least important

def feature_importance(model, data):
    """
    Function to show which features are most important in the model.
    ::param_model:: Which model to use?
    ::param_data:: What data to use?
    """
    fea_imp = pd.DataFrame({'imp': model.feature_importances_, 'col': data.columns})
    fea_imp = fea_imp.sort_values(['imp', 'col'], ascending=[True, False]).iloc[-30:]
    _= fea_imp.plot(kind='barh', x='col', y='imp', figsize=(20, 10))
    return fea_imp


feature_importance(catboost_model, X_train)

	imp	col
11	1.213266	forecast_price_pow_off_peak
3	1.254589	cons_gas_12m
6	1.352416	forecast_cons_year
13	1.576614	imp_cons
10	1.934195	forecast_price_energy_peak
1	1.992084	channel_sales
23	2.070375	year_date_activ
18	2.143501	num_years_antig
31	2.187367	day_date_modif_prod
30	2.310598	month_date_modif_prod
24	2.418985	month_date_activ
19	2.483177	origin_up
33	2.513828	month_date_renewal
34	2.528226	day_date_renewal
29	2.547458	year_date_modif_prod
28	2.548470	day_date_end
25	2.916687	day_date_activ
22	3.074912	fore_casted_price_energy_total
9	3.084097	forecast_price_energy_off_peak
27	3.674631	month_date_end
5	4.124847	forecast_cons_12m
2	4.349512	cons_12m
0	4.481502	id
17	4.542184	net_margin
21	4.784779	total_cons_12m
4	5.258148	cons_last_month
8	5.543921	forecast_meter_rent_12m
15	5.792107	margin_net_pow_ele
14	5.925329	margin_gross_pow_ele
20	7.133139	pow_max

As we can see for this model we can clearly notice that we have four features with importances under 1 comapred to most of the features ranging between 3 and 7.5

# X_train.drop(columns=["has_gas","nb_prod_act","num_years_antig","forecast_discount_energy"], inplace=True)

# X_test.drop(columns=["has_gas","nb_prod_act","num_years_antig","forecast_discount_energy"], inplace=True)

Modelling

Now let’s evaluate how well this trained model is able to predict the values of the test dataset. We are going to use 3 metrics to evaluate performance:
• Accuracy = the ratio of correctly predicted observations to the total observations
• Precision = the ability of the classifier to not label a negative sample as positive
• Recall = the ability of the classifier to find all the positive samples
• F1-score: a way of combining the precision and recall of the model, and it is defined as the harmonic mean of the model’s precision and recall.
• AUC : the measure of the ability of the classifier to distinguish between classes and is used as a summary of the ROC curve. The higher the AUC, the better the performance of the model at distinguishing between the positive and negative classes.

Let's begin by training our Catboost model and testing it on the test dataset and see how well it performs

train_pool = Pool(X_train,y_train)

# CatBoost model definition
catboost_model = CatBoostClassifier(
                                    iterations=1000,
                                    custom_loss=['Accuracy'],
                                    loss_function='CrossEntropy',depth=5
                                   )

# Fit CatBoost model
catboost_model.fit(train_pool,
                   plot=True)

# CatBoost accuracy
acc_catboost = round(catboost_model.score(X_train, y_train) * 100, 2)

MetricVisualizer(layout=Layout(align_self='stretch', height='500px'))

0:	learn: 0.6659824	total: 25.8ms	remaining: 25.8s
1:	learn: 0.6405579	total: 34.3ms	remaining: 17.1s
2:	learn: 0.6175952	total: 42.9ms	remaining: 14.3s
3:	learn: 0.5957692	total: 50.9ms	remaining: 12.7s
4:	learn: 0.5753977	total: 58.8ms	remaining: 11.7s
5:	learn: 0.5566162	total: 66.6ms	remaining: 11s
6:	learn: 0.5390072	total: 74.3ms	remaining: 10.5s
7:	learn: 0.5232824	total: 81.9ms	remaining: 10.2s
8:	learn: 0.5078731	total: 89.8ms	remaining: 9.88s
9:	learn: 0.4942584	total: 97.5ms	remaining: 9.65s
10:	learn: 0.4817198	total: 105ms	remaining: 9.45s
11:	learn: 0.4701478	total: 113ms	remaining: 9.29s
12:	learn: 0.4589232	total: 120ms	remaining: 9.14s
13:	learn: 0.4487878	total: 128ms	remaining: 9.02s
14:	learn: 0.4394818	total: 136ms	remaining: 8.92s
15:	learn: 0.4297546	total: 144ms	remaining: 8.83s
16:	learn: 0.4214325	total: 151ms	remaining: 8.74s
17:	learn: 0.4141087	total: 159ms	remaining: 8.66s
18:	learn: 0.4072672	total: 167ms	remaining: 8.6s
19:	learn: 0.4005293	total: 174ms	remaining: 8.55s
20:	learn: 0.3946043	total: 182ms	remaining: 8.49s
21:	learn: 0.3885827	total: 190ms	remaining: 8.44s
22:	learn: 0.3832021	total: 198ms	remaining: 8.4s
23:	learn: 0.3783827	total: 205ms	remaining: 8.35s
24:	learn: 0.3735162	total: 213ms	remaining: 8.31s
25:	learn: 0.3693656	total: 222ms	remaining: 8.3s
26:	learn: 0.3654390	total: 230ms	remaining: 8.29s
27:	learn: 0.3615553	total: 238ms	remaining: 8.27s
28:	learn: 0.3580022	total: 246ms	remaining: 8.25s
29:	learn: 0.3543967	total: 254ms	remaining: 8.23s
30:	learn: 0.3511688	total: 262ms	remaining: 8.2s
31:	learn: 0.3480313	total: 270ms	remaining: 8.17s
32:	learn: 0.3455761	total: 278ms	remaining: 8.14s
33:	learn: 0.3433082	total: 286ms	remaining: 8.11s
34:	learn: 0.3405098	total: 294ms	remaining: 8.1s
35:	learn: 0.3382171	total: 302ms	remaining: 8.08s
36:	learn: 0.3362031	total: 309ms	remaining: 8.05s
37:	learn: 0.3341666	total: 317ms	remaining: 8.02s
38:	learn: 0.3324831	total: 324ms	remaining: 7.99s
39:	learn: 0.3308527	total: 332ms	remaining: 7.97s
40:	learn: 0.3292917	total: 340ms	remaining: 7.95s
41:	learn: 0.3278054	total: 347ms	remaining: 7.92s
42:	learn: 0.3261658	total: 355ms	remaining: 7.9s
43:	learn: 0.3248788	total: 363ms	remaining: 7.88s
44:	learn: 0.3233453	total: 370ms	remaining: 7.86s
45:	learn: 0.3222141	total: 378ms	remaining: 7.84s
46:	learn: 0.3208569	total: 385ms	remaining: 7.81s
47:	learn: 0.3196304	total: 393ms	remaining: 7.8s
48:	learn: 0.3186704	total: 401ms	remaining: 7.78s
49:	learn: 0.3174357	total: 408ms	remaining: 7.76s
50:	learn: 0.3164986	total: 417ms	remaining: 7.75s
51:	learn: 0.3154912	total: 425ms	remaining: 7.75s
52:	learn: 0.3146420	total: 433ms	remaining: 7.74s
53:	learn: 0.3139497	total: 441ms	remaining: 7.72s
54:	learn: 0.3133423	total: 449ms	remaining: 7.71s
55:	learn: 0.3125126	total: 458ms	remaining: 7.71s
56:	learn: 0.3119221	total: 466ms	remaining: 7.71s
57:	learn: 0.3111901	total: 474ms	remaining: 7.7s
58:	learn: 0.3106373	total: 482ms	remaining: 7.68s
59:	learn: 0.3099091	total: 490ms	remaining: 7.67s
60:	learn: 0.3092748	total: 498ms	remaining: 7.66s
61:	learn: 0.3086159	total: 506ms	remaining: 7.65s
62:	learn: 0.3081743	total: 513ms	remaining: 7.63s
63:	learn: 0.3073849	total: 521ms	remaining: 7.62s
64:	learn: 0.3069738	total: 529ms	remaining: 7.61s
65:	learn: 0.3066172	total: 537ms	remaining: 7.6s
66:	learn: 0.3061270	total: 545ms	remaining: 7.59s
67:	learn: 0.3056199	total: 553ms	remaining: 7.57s
68:	learn: 0.3051018	total: 560ms	remaining: 7.56s
69:	learn: 0.3044529	total: 568ms	remaining: 7.54s
70:	learn: 0.3040192	total: 575ms	remaining: 7.53s
71:	learn: 0.3034136	total: 583ms	remaining: 7.52s
72:	learn: 0.3030527	total: 591ms	remaining: 7.5s
73:	learn: 0.3027174	total: 599ms	remaining: 7.5s
74:	learn: 0.3024280	total: 608ms	remaining: 7.5s
75:	learn: 0.3020214	total: 616ms	remaining: 7.49s
76:	learn: 0.3016066	total: 624ms	remaining: 7.48s
77:	learn: 0.3013100	total: 632ms	remaining: 7.47s
78:	learn: 0.3008993	total: 640ms	remaining: 7.46s
79:	learn: 0.3004685	total: 648ms	remaining: 7.45s
80:	learn: 0.3002172	total: 656ms	remaining: 7.44s
81:	learn: 0.2999444	total: 664ms	remaining: 7.43s
82:	learn: 0.2995666	total: 671ms	remaining: 7.42s
83:	learn: 0.2993016	total: 679ms	remaining: 7.41s
84:	learn: 0.2988826	total: 687ms	remaining: 7.39s
85:	learn: 0.2986351	total: 695ms	remaining: 7.38s
86:	learn: 0.2983863	total: 702ms	remaining: 7.37s
87:	learn: 0.2980171	total: 710ms	remaining: 7.36s
88:	learn: 0.2977403	total: 718ms	remaining: 7.35s
89:	learn: 0.2974953	total: 726ms	remaining: 7.34s
90:	learn: 0.2973102	total: 733ms	remaining: 7.32s
91:	learn: 0.2969956	total: 741ms	remaining: 7.31s
92:	learn: 0.2967368	total: 749ms	remaining: 7.3s
93:	learn: 0.2965637	total: 756ms	remaining: 7.29s
94:	learn: 0.2963405	total: 764ms	remaining: 7.28s
95:	learn: 0.2961521	total: 772ms	remaining: 7.27s
96:	learn: 0.2958792	total: 779ms	remaining: 7.25s
97:	learn: 0.2956809	total: 788ms	remaining: 7.25s
98:	learn: 0.2954239	total: 796ms	remaining: 7.25s
99:	learn: 0.2951342	total: 805ms	remaining: 7.24s
100:	learn: 0.2948867	total: 813ms	remaining: 7.24s
101:	learn: 0.2946644	total: 821ms	remaining: 7.23s
102:	learn: 0.2942808	total: 829ms	remaining: 7.22s
103:	learn: 0.2940046	total: 838ms	remaining: 7.22s
104:	learn: 0.2938677	total: 846ms	remaining: 7.21s
105:	learn: 0.2935676	total: 853ms	remaining: 7.2s
106:	learn: 0.2931513	total: 861ms	remaining: 7.19s
107:	learn: 0.2929200	total: 869ms	remaining: 7.18s
108:	learn: 0.2926363	total: 877ms	remaining: 7.17s
109:	learn: 0.2925167	total: 885ms	remaining: 7.16s
110:	learn: 0.2922955	total: 893ms	remaining: 7.15s
111:	learn: 0.2921320	total: 900ms	remaining: 7.14s
112:	learn: 0.2919454	total: 908ms	remaining: 7.13s
113:	learn: 0.2918265	total: 916ms	remaining: 7.12s
114:	learn: 0.2915485	total: 923ms	remaining: 7.11s
115:	learn: 0.2913738	total: 931ms	remaining: 7.09s
116:	learn: 0.2912518	total: 939ms	remaining: 7.09s
117:	learn: 0.2911091	total: 947ms	remaining: 7.08s
118:	learn: 0.2908786	total: 954ms	remaining: 7.07s
119:	learn: 0.2906676	total: 962ms	remaining: 7.05s
120:	learn: 0.2905568	total: 970ms	remaining: 7.04s
121:	learn: 0.2903138	total: 978ms	remaining: 7.04s
122:	learn: 0.2901788	total: 993ms	remaining: 7.08s
123:	learn: 0.2900386	total: 1s	remaining: 7.08s
124:	learn: 0.2899042	total: 1.01s	remaining: 7.08s
125:	learn: 0.2897820	total: 1.02s	remaining: 7.07s
126:	learn: 0.2895897	total: 1.03s	remaining: 7.07s
127:	learn: 0.2894179	total: 1.04s	remaining: 7.06s
128:	learn: 0.2891133	total: 1.04s	remaining: 7.06s
129:	learn: 0.2889714	total: 1.05s	remaining: 7.05s
130:	learn: 0.2888294	total: 1.06s	remaining: 7.05s
131:	learn: 0.2885993	total: 1.07s	remaining: 7.04s
132:	learn: 0.2884202	total: 1.08s	remaining: 7.04s
133:	learn: 0.2882644	total: 1.09s	remaining: 7.03s
134:	learn: 0.2880881	total: 1.1s	remaining: 7.03s
135:	learn: 0.2879370	total: 1.1s	remaining: 7.02s
136:	learn: 0.2877882	total: 1.11s	remaining: 7.01s
137:	learn: 0.2875292	total: 1.12s	remaining: 7.01s
138:	learn: 0.2874122	total: 1.13s	remaining: 7s
139:	learn: 0.2872162	total: 1.14s	remaining: 6.99s
140:	learn: 0.2868727	total: 1.15s	remaining: 6.98s
141:	learn: 0.2866856	total: 1.15s	remaining: 6.97s
142:	learn: 0.2864487	total: 1.16s	remaining: 6.96s
143:	learn: 0.2862735	total: 1.17s	remaining: 6.95s
144:	learn: 0.2861235	total: 1.18s	remaining: 6.94s
145:	learn: 0.2858975	total: 1.19s	remaining: 6.93s
146:	learn: 0.2857673	total: 1.19s	remaining: 6.93s
147:	learn: 0.2855723	total: 1.2s	remaining: 6.92s
148:	learn: 0.2854244	total: 1.22s	remaining: 6.94s
149:	learn: 0.2853219	total: 1.22s	remaining: 6.94s
150:	learn: 0.2852068	total: 1.23s	remaining: 6.93s
151:	learn: 0.2850253	total: 1.24s	remaining: 6.93s
152:	learn: 0.2848283	total: 1.25s	remaining: 6.92s
153:	learn: 0.2847084	total: 1.26s	remaining: 6.91s
154:	learn: 0.2845471	total: 1.27s	remaining: 6.9s
155:	learn: 0.2843973	total: 1.27s	remaining: 6.89s
156:	learn: 0.2842693	total: 1.28s	remaining: 6.88s
157:	learn: 0.2841483	total: 1.29s	remaining: 6.88s
158:	learn: 0.2839903	total: 1.3s	remaining: 6.87s
159:	learn: 0.2838014	total: 1.31s	remaining: 6.87s
160:	learn: 0.2836920	total: 1.31s	remaining: 6.86s
161:	learn: 0.2835902	total: 1.32s	remaining: 6.85s
162:	learn: 0.2834765	total: 1.33s	remaining: 6.84s
163:	learn: 0.2833687	total: 1.34s	remaining: 6.83s
164:	learn: 0.2831634	total: 1.35s	remaining: 6.82s
165:	learn: 0.2830367	total: 1.36s	remaining: 6.82s
166:	learn: 0.2829258	total: 1.36s	remaining: 6.81s
167:	learn: 0.2827520	total: 1.37s	remaining: 6.8s
168:	learn: 0.2826376	total: 1.38s	remaining: 6.79s
169:	learn: 0.2825152	total: 1.39s	remaining: 6.79s
170:	learn: 0.2823954	total: 1.4s	remaining: 6.78s
171:	learn: 0.2822768	total: 1.41s	remaining: 6.76s
172:	learn: 0.2820453	total: 1.41s	remaining: 6.76s
173:	learn: 0.2819167	total: 1.42s	remaining: 6.75s
174:	learn: 0.2817064	total: 1.43s	remaining: 6.74s
175:	learn: 0.2815446	total: 1.44s	remaining: 6.74s
176:	learn: 0.2813904	total: 1.45s	remaining: 6.73s
177:	learn: 0.2812088	total: 1.46s	remaining: 6.73s
178:	learn: 0.2810681	total: 1.47s	remaining: 6.72s
179:	learn: 0.2806865	total: 1.47s	remaining: 6.71s
180:	learn: 0.2805176	total: 1.48s	remaining: 6.71s
181:	learn: 0.2804030	total: 1.49s	remaining: 6.7s
182:	learn: 0.2802564	total: 1.5s	remaining: 6.69s
183:	learn: 0.2800065	total: 1.51s	remaining: 6.68s
184:	learn: 0.2798893	total: 1.51s	remaining: 6.67s
185:	learn: 0.2797287	total: 1.52s	remaining: 6.66s
186:	learn: 0.2795906	total: 1.53s	remaining: 6.65s
187:	learn: 0.2794678	total: 1.54s	remaining: 6.64s
188:	learn: 0.2793049	total: 1.55s	remaining: 6.64s
189:	learn: 0.2791951	total: 1.55s	remaining: 6.63s
190:	learn: 0.2790785	total: 1.56s	remaining: 6.62s
191:	learn: 0.2789654	total: 1.57s	remaining: 6.62s
192:	learn: 0.2789293	total: 1.58s	remaining: 6.61s
193:	learn: 0.2788345	total: 1.59s	remaining: 6.6s
194:	learn: 0.2786405	total: 1.6s	remaining: 6.59s
195:	learn: 0.2784205	total: 1.6s	remaining: 6.58s
196:	learn: 0.2783140	total: 1.61s	remaining: 6.58s
197:	learn: 0.2781797	total: 1.62s	remaining: 6.57s
198:	learn: 0.2780585	total: 1.63s	remaining: 6.56s
199:	learn: 0.2779278	total: 1.64s	remaining: 6.55s
200:	learn: 0.2778252	total: 1.65s	remaining: 6.54s
201:	learn: 0.2777250	total: 1.65s	remaining: 6.53s
202:	learn: 0.2776351	total: 1.66s	remaining: 6.53s
203:	learn: 0.2774977	total: 1.67s	remaining: 6.52s
204:	learn: 0.2773676	total: 1.68s	remaining: 6.51s
205:	learn: 0.2772591	total: 1.69s	remaining: 6.5s
206:	learn: 0.2771230	total: 1.69s	remaining: 6.49s
207:	learn: 0.2769973	total: 1.7s	remaining: 6.49s
208:	learn: 0.2768886	total: 1.71s	remaining: 6.48s
209:	learn: 0.2767459	total: 1.72s	remaining: 6.47s
210:	learn: 0.2766523	total: 1.73s	remaining: 6.46s
211:	learn: 0.2765695	total: 1.74s	remaining: 6.45s
212:	learn: 0.2763649	total: 1.75s	remaining: 6.45s
213:	learn: 0.2762245	total: 1.75s	remaining: 6.44s
214:	learn: 0.2761119	total: 1.76s	remaining: 6.43s
215:	learn: 0.2760007	total: 1.77s	remaining: 6.42s
216:	learn: 0.2759013	total: 1.78s	remaining: 6.41s
217:	learn: 0.2757549	total: 1.78s	remaining: 6.4s
218:	learn: 0.2756298	total: 1.79s	remaining: 6.39s
219:	learn: 0.2754923	total: 1.8s	remaining: 6.38s
220:	learn: 0.2753516	total: 1.81s	remaining: 6.37s
221:	learn: 0.2752650	total: 1.81s	remaining: 6.36s
222:	learn: 0.2751821	total: 1.82s	remaining: 6.35s
223:	learn: 0.2750776	total: 1.83s	remaining: 6.34s
224:	learn: 0.2749463	total: 1.84s	remaining: 6.33s
225:	learn: 0.2748100	total: 1.85s	remaining: 6.32s
226:	learn: 0.2746633	total: 1.85s	remaining: 6.31s
227:	learn: 0.2745782	total: 1.86s	remaining: 6.3s
228:	learn: 0.2744863	total: 1.87s	remaining: 6.29s
229:	learn: 0.2744245	total: 1.88s	remaining: 6.28s
230:	learn: 0.2742987	total: 1.88s	remaining: 6.27s
231:	learn: 0.2741001	total: 1.89s	remaining: 6.26s
232:	learn: 0.2739773	total: 1.9s	remaining: 6.25s
233:	learn: 0.2738003	total: 1.91s	remaining: 6.25s
234:	learn: 0.2736854	total: 1.92s	remaining: 6.24s
235:	learn: 0.2733932	total: 1.93s	remaining: 6.24s
236:	learn: 0.2732870	total: 1.93s	remaining: 6.23s
237:	learn: 0.2730978	total: 1.94s	remaining: 6.22s
238:	learn: 0.2729357	total: 1.95s	remaining: 6.21s
239:	learn: 0.2728556	total: 1.96s	remaining: 6.2s
240:	learn: 0.2726577	total: 1.97s	remaining: 6.19s
241:	learn: 0.2725177	total: 1.97s	remaining: 6.18s
242:	learn: 0.2724568	total: 1.98s	remaining: 6.17s
243:	learn: 0.2723783	total: 2s	remaining: 6.19s
244:	learn: 0.2722647	total: 2s	remaining: 6.18s
245:	learn: 0.2721266	total: 2.01s	remaining: 6.17s
246:	learn: 0.2719629	total: 2.02s	remaining: 6.16s
247:	learn: 0.2718415	total: 2.03s	remaining: 6.15s
248:	learn: 0.2717672	total: 2.04s	remaining: 6.14s
249:	learn: 0.2716562	total: 2.04s	remaining: 6.13s
250:	learn: 0.2715273	total: 2.05s	remaining: 6.12s
251:	learn: 0.2713233	total: 2.06s	remaining: 6.11s
252:	learn: 0.2711585	total: 2.07s	remaining: 6.1s
253:	learn: 0.2710264	total: 2.07s	remaining: 6.09s
254:	learn: 0.2709238	total: 2.08s	remaining: 6.08s
255:	learn: 0.2707380	total: 2.09s	remaining: 6.07s
256:	learn: 0.2706147	total: 2.1s	remaining: 6.07s
257:	learn: 0.2705446	total: 2.11s	remaining: 6.06s
258:	learn: 0.2704140	total: 2.12s	remaining: 6.05s
259:	learn: 0.2702277	total: 2.12s	remaining: 6.04s
260:	learn: 0.2700105	total: 2.13s	remaining: 6.04s
261:	learn: 0.2698879	total: 2.14s	remaining: 6.03s
262:	learn: 0.2698188	total: 2.15s	remaining: 6.02s
263:	learn: 0.2697322	total: 2.15s	remaining: 6.01s
264:	learn: 0.2695722	total: 2.16s	remaining: 6s
265:	learn: 0.2694767	total: 2.17s	remaining: 5.99s
266:	learn: 0.2693831	total: 2.18s	remaining: 5.98s
267:	learn: 0.2691713	total: 2.19s	remaining: 5.97s
268:	learn: 0.2690294	total: 2.19s	remaining: 5.96s
269:	learn: 0.2689518	total: 2.2s	remaining: 5.95s
270:	learn: 0.2687903	total: 2.21s	remaining: 5.94s
271:	learn: 0.2686981	total: 2.22s	remaining: 5.93s
272:	learn: 0.2686030	total: 2.22s	remaining: 5.92s
273:	learn: 0.2683641	total: 2.23s	remaining: 5.91s
274:	learn: 0.2682184	total: 2.24s	remaining: 5.9s
275:	learn: 0.2681323	total: 2.25s	remaining: 5.89s
276:	learn: 0.2679967	total: 2.25s	remaining: 5.88s
277:	learn: 0.2679201	total: 2.26s	remaining: 5.88s
278:	learn: 0.2677401	total: 2.27s	remaining: 5.87s
279:	learn: 0.2674958	total: 2.28s	remaining: 5.86s
280:	learn: 0.2673855	total: 2.29s	remaining: 5.85s
281:	learn: 0.2672433	total: 2.29s	remaining: 5.84s
282:	learn: 0.2671174	total: 2.3s	remaining: 5.83s
283:	learn: 0.2669682	total: 2.31s	remaining: 5.83s
284:	learn: 0.2667672	total: 2.32s	remaining: 5.82s
285:	learn: 0.2666044	total: 2.33s	remaining: 5.81s
286:	learn: 0.2664981	total: 2.33s	remaining: 5.8s
287:	learn: 0.2663759	total: 2.34s	remaining: 5.79s
288:	learn: 0.2662160	total: 2.35s	remaining: 5.78s
289:	learn: 0.2661149	total: 2.36s	remaining: 5.77s
290:	learn: 0.2659153	total: 2.36s	remaining: 5.76s
291:	learn: 0.2657842	total: 2.37s	remaining: 5.75s
292:	learn: 0.2657279	total: 2.38s	remaining: 5.74s
293:	learn: 0.2655767	total: 2.39s	remaining: 5.73s
294:	learn: 0.2654857	total: 2.4s	remaining: 5.72s
295:	learn: 0.2653991	total: 2.4s	remaining: 5.72s
296:	learn: 0.2653154	total: 2.41s	remaining: 5.71s
297:	learn: 0.2652201	total: 2.42s	remaining: 5.7s
298:	learn: 0.2651340	total: 2.43s	remaining: 5.7s
299:	learn: 0.2650726	total: 2.44s	remaining: 5.69s
300:	learn: 0.2648814	total: 2.45s	remaining: 5.68s
301:	learn: 0.2647603	total: 2.46s	remaining: 5.67s
302:	learn: 0.2646444	total: 2.46s	remaining: 5.67s
303:	learn: 0.2645222	total: 2.47s	remaining: 5.66s
304:	learn: 0.2644040	total: 2.48s	remaining: 5.65s
305:	learn: 0.2642479	total: 2.49s	remaining: 5.64s
306:	learn: 0.2641182	total: 2.5s	remaining: 5.63s
307:	learn: 0.2639839	total: 2.5s	remaining: 5.63s
308:	learn: 0.2638260	total: 2.51s	remaining: 5.62s
309:	learn: 0.2637610	total: 2.52s	remaining: 5.61s
310:	learn: 0.2636361	total: 2.53s	remaining: 5.6s
311:	learn: 0.2635208	total: 2.54s	remaining: 5.59s
312:	learn: 0.2633632	total: 2.54s	remaining: 5.58s
313:	learn: 0.2632738	total: 2.55s	remaining: 5.58s
314:	learn: 0.2632013	total: 2.56s	remaining: 5.57s
315:	learn: 0.2630672	total: 2.57s	remaining: 5.56s
316:	learn: 0.2629070	total: 2.57s	remaining: 5.55s
317:	learn: 0.2627791	total: 2.58s	remaining: 5.54s
318:	learn: 0.2626569	total: 2.59s	remaining: 5.53s
319:	learn: 0.2624693	total: 2.6s	remaining: 5.52s
320:	learn: 0.2623654	total: 2.6s	remaining: 5.51s
321:	learn: 0.2621780	total: 2.61s	remaining: 5.5s
322:	learn: 0.2620440	total: 2.62s	remaining: 5.49s
323:	learn: 0.2619203	total: 2.63s	remaining: 5.48s
324:	learn: 0.2617852	total: 2.64s	remaining: 5.47s
325:	learn: 0.2616589	total: 2.64s	remaining: 5.47s
326:	learn: 0.2615293	total: 2.65s	remaining: 5.46s
327:	learn: 0.2614303	total: 2.66s	remaining: 5.45s
328:	learn: 0.2613447	total: 2.67s	remaining: 5.44s
329:	learn: 0.2612439	total: 2.68s	remaining: 5.44s
330:	learn: 0.2611746	total: 2.69s	remaining: 5.43s
331:	learn: 0.2610849	total: 2.69s	remaining: 5.42s
332:	learn: 0.2609968	total: 2.7s	remaining: 5.41s
333:	learn: 0.2608563	total: 2.71s	remaining: 5.4s
334:	learn: 0.2607413	total: 2.72s	remaining: 5.39s
335:	learn: 0.2605412	total: 2.73s	remaining: 5.38s
336:	learn: 0.2604597	total: 2.73s	remaining: 5.38s
337:	learn: 0.2603115	total: 2.74s	remaining: 5.37s
338:	learn: 0.2602584	total: 2.75s	remaining: 5.36s
339:	learn: 0.2601805	total: 2.76s	remaining: 5.35s
340:	learn: 0.2600239	total: 2.76s	remaining: 5.34s
341:	learn: 0.2599117	total: 2.77s	remaining: 5.33s
342:	learn: 0.2597976	total: 2.78s	remaining: 5.33s
343:	learn: 0.2596856	total: 2.79s	remaining: 5.32s
344:	learn: 0.2594723	total: 2.79s	remaining: 5.31s
345:	learn: 0.2593120	total: 2.8s	remaining: 5.3s
346:	learn: 0.2591039	total: 2.81s	remaining: 5.29s
347:	learn: 0.2589492	total: 2.82s	remaining: 5.28s
348:	learn: 0.2587373	total: 2.83s	remaining: 5.27s
349:	learn: 0.2585984	total: 2.83s	remaining: 5.26s
350:	learn: 0.2584271	total: 2.84s	remaining: 5.25s
351:	learn: 0.2582818	total: 2.85s	remaining: 5.25s
352:	learn: 0.2581235	total: 2.86s	remaining: 5.24s
353:	learn: 0.2579898	total: 2.87s	remaining: 5.23s
354:	learn: 0.2578892	total: 2.88s	remaining: 5.22s
355:	learn: 0.2577189	total: 2.88s	remaining: 5.22s
356:	learn: 0.2575068	total: 2.89s	remaining: 5.21s
357:	learn: 0.2573273	total: 2.9s	remaining: 5.2s
358:	learn: 0.2572316	total: 2.91s	remaining: 5.19s
359:	learn: 0.2571339	total: 2.92s	remaining: 5.18s
360:	learn: 0.2570391	total: 2.92s	remaining: 5.17s
361:	learn: 0.2569610	total: 2.93s	remaining: 5.17s
362:	learn: 0.2568010	total: 2.94s	remaining: 5.16s
363:	learn: 0.2565908	total: 2.95s	remaining: 5.15s
364:	learn: 0.2564256	total: 2.96s	remaining: 5.14s
365:	learn: 0.2562727	total: 2.96s	remaining: 5.13s
366:	learn: 0.2561406	total: 2.97s	remaining: 5.13s
367:	learn: 0.2560296	total: 2.98s	remaining: 5.12s
368:	learn: 0.2559550	total: 3s	remaining: 5.13s
369:	learn: 0.2558184	total: 3.01s	remaining: 5.12s
370:	learn: 0.2556781	total: 3.02s	remaining: 5.11s
371:	learn: 0.2555820	total: 3.02s	remaining: 5.11s
372:	learn: 0.2555244	total: 3.03s	remaining: 5.1s
373:	learn: 0.2553953	total: 3.04s	remaining: 5.09s
374:	learn: 0.2552735	total: 3.05s	remaining: 5.08s
375:	learn: 0.2551532	total: 3.06s	remaining: 5.07s
376:	learn: 0.2550259	total: 3.06s	remaining: 5.07s
377:	learn: 0.2548621	total: 3.07s	remaining: 5.06s
378:	learn: 0.2547441	total: 3.08s	remaining: 5.05s
379:	learn: 0.2545757	total: 3.09s	remaining: 5.04s
380:	learn: 0.2544704	total: 3.1s	remaining: 5.03s
381:	learn: 0.2543879	total: 3.1s	remaining: 5.02s
382:	learn: 0.2542961	total: 3.11s	remaining: 5.01s
383:	learn: 0.2542287	total: 3.12s	remaining: 5s
384:	learn: 0.2540925	total: 3.13s	remaining: 5s
385:	learn: 0.2539513	total: 3.13s	remaining: 4.99s
386:	learn: 0.2538085	total: 3.14s	remaining: 4.98s
387:	learn: 0.2536336	total: 3.15s	remaining: 4.97s
388:	learn: 0.2535215	total: 3.16s	remaining: 4.96s
389:	learn: 0.2534249	total: 3.17s	remaining: 4.95s
390:	learn: 0.2533198	total: 3.17s	remaining: 4.94s
391:	learn: 0.2532631	total: 3.18s	remaining: 4.93s
392:	learn: 0.2531830	total: 3.19s	remaining: 4.92s
393:	learn: 0.2530726	total: 3.19s	remaining: 4.92s
394:	learn: 0.2529418	total: 3.2s	remaining: 4.91s
395:	learn: 0.2528359	total: 3.21s	remaining: 4.9s
396:	learn: 0.2526901	total: 3.22s	remaining: 4.89s
397:	learn: 0.2525773	total: 3.23s	remaining: 4.88s
398:	learn: 0.2524341	total: 3.24s	remaining: 4.88s
399:	learn: 0.2522729	total: 3.25s	remaining: 4.87s
400:	learn: 0.2520926	total: 3.25s	remaining: 4.86s
401:	learn: 0.2519695	total: 3.26s	remaining: 4.85s
402:	learn: 0.2518781	total: 3.27s	remaining: 4.84s
403:	learn: 0.2517732	total: 3.28s	remaining: 4.83s
404:	learn: 0.2516661	total: 3.28s	remaining: 4.82s
405:	learn: 0.2515361	total: 3.29s	remaining: 4.82s
406:	learn: 0.2514157	total: 3.3s	remaining: 4.81s
407:	learn: 0.2513393	total: 3.31s	remaining: 4.8s
408:	learn: 0.2512702	total: 3.31s	remaining: 4.79s
409:	learn: 0.2511839	total: 3.32s	remaining: 4.78s
410:	learn: 0.2510706	total: 3.33s	remaining: 4.77s
411:	learn: 0.2509466	total: 3.34s	remaining: 4.76s
412:	learn: 0.2508211	total: 3.35s	remaining: 4.75s
413:	learn: 0.2506888	total: 3.35s	remaining: 4.75s
414:	learn: 0.2505854	total: 3.36s	remaining: 4.74s
415:	learn: 0.2504620	total: 3.37s	remaining: 4.73s
416:	learn: 0.2503664	total: 3.38s	remaining: 4.72s
417:	learn: 0.2502433	total: 3.38s	remaining: 4.71s
418:	learn: 0.2501237	total: 3.39s	remaining: 4.7s
419:	learn: 0.2500042	total: 3.4s	remaining: 4.7s
420:	learn: 0.2498734	total: 3.41s	remaining: 4.69s
421:	learn: 0.2497835	total: 3.42s	remaining: 4.68s
422:	learn: 0.2496922	total: 3.43s	remaining: 4.67s
423:	learn: 0.2495876	total: 3.43s	remaining: 4.66s
424:	learn: 0.2495088	total: 3.44s	remaining: 4.66s
425:	learn: 0.2494215	total: 3.45s	remaining: 4.65s
426:	learn: 0.2493497	total: 3.46s	remaining: 4.64s
427:	learn: 0.2492174	total: 3.47s	remaining: 4.63s
428:	learn: 0.2491344	total: 3.47s	remaining: 4.62s
429:	learn: 0.2490369	total: 3.48s	remaining: 4.62s
430:	learn: 0.2489147	total: 3.49s	remaining: 4.61s
431:	learn: 0.2487584	total: 3.5s	remaining: 4.6s
432:	learn: 0.2486465	total: 3.51s	remaining: 4.59s
433:	learn: 0.2485167	total: 3.51s	remaining: 4.58s
434:	learn: 0.2484209	total: 3.52s	remaining: 4.57s
435:	learn: 0.2483421	total: 3.53s	remaining: 4.57s
436:	learn: 0.2482036	total: 3.54s	remaining: 4.56s
437:	learn: 0.2480811	total: 3.54s	remaining: 4.55s
438:	learn: 0.2479224	total: 3.55s	remaining: 4.54s
439:	learn: 0.2478263	total: 3.56s	remaining: 4.53s
440:	learn: 0.2477593	total: 3.57s	remaining: 4.52s
441:	learn: 0.2476440	total: 3.57s	remaining: 4.51s
442:	learn: 0.2475507	total: 3.58s	remaining: 4.5s
443:	learn: 0.2474416	total: 3.59s	remaining: 4.5s
444:	learn: 0.2473228	total: 3.6s	remaining: 4.49s
445:	learn: 0.2472174	total: 3.61s	remaining: 4.48s
446:	learn: 0.2471051	total: 3.61s	remaining: 4.47s
447:	learn: 0.2470036	total: 3.62s	remaining: 4.46s
448:	learn: 0.2468908	total: 3.63s	remaining: 4.46s
449:	learn: 0.2467059	total: 3.64s	remaining: 4.45s
450:	learn: 0.2466150	total: 3.65s	remaining: 4.44s
451:	learn: 0.2464962	total: 3.65s	remaining: 4.43s
452:	learn: 0.2464163	total: 3.66s	remaining: 4.42s
453:	learn: 0.2462692	total: 3.67s	remaining: 4.41s
454:	learn: 0.2461763	total: 3.68s	remaining: 4.4s
455:	learn: 0.2460744	total: 3.68s	remaining: 4.39s
456:	learn: 0.2459618	total: 3.69s	remaining: 4.39s
457:	learn: 0.2458762	total: 3.7s	remaining: 4.38s
458:	learn: 0.2457692	total: 3.71s	remaining: 4.37s
459:	learn: 0.2456518	total: 3.71s	remaining: 4.36s
460:	learn: 0.2455437	total: 3.72s	remaining: 4.35s
461:	learn: 0.2454920	total: 3.73s	remaining: 4.34s
462:	learn: 0.2453862	total: 3.74s	remaining: 4.33s
463:	learn: 0.2452353	total: 3.75s	remaining: 4.33s
464:	learn: 0.2450947	total: 3.75s	remaining: 4.32s
465:	learn: 0.2449827	total: 3.76s	remaining: 4.31s
466:	learn: 0.2448741	total: 3.77s	remaining: 4.3s
467:	learn: 0.2447763	total: 3.78s	remaining: 4.29s
468:	learn: 0.2446784	total: 3.79s	remaining: 4.29s
469:	learn: 0.2446025	total: 3.79s	remaining: 4.28s
470:	learn: 0.2444371	total: 3.8s	remaining: 4.27s
471:	learn: 0.2443171	total: 3.81s	remaining: 4.26s
472:	learn: 0.2442051	total: 3.82s	remaining: 4.25s
473:	learn: 0.2441010	total: 3.82s	remaining: 4.24s
474:	learn: 0.2439982	total: 3.83s	remaining: 4.24s
475:	learn: 0.2439315	total: 3.84s	remaining: 4.23s
476:	learn: 0.2438278	total: 3.85s	remaining: 4.22s
477:	learn: 0.2437019	total: 3.85s	remaining: 4.21s
478:	learn: 0.2436112	total: 3.86s	remaining: 4.2s
479:	learn: 0.2435340	total: 3.87s	remaining: 4.19s
480:	learn: 0.2434416	total: 3.88s	remaining: 4.18s
481:	learn: 0.2433807	total: 3.89s	remaining: 4.18s
482:	learn: 0.2432707	total: 3.89s	remaining: 4.17s
483:	learn: 0.2431799	total: 3.9s	remaining: 4.16s
484:	learn: 0.2430503	total: 3.91s	remaining: 4.15s
485:	learn: 0.2429439	total: 3.92s	remaining: 4.14s
486:	learn: 0.2428296	total: 3.93s	remaining: 4.14s
487:	learn: 0.2427242	total: 3.93s	remaining: 4.13s
488:	learn: 0.2426635	total: 3.94s	remaining: 4.12s
489:	learn: 0.2426078	total: 3.95s	remaining: 4.11s
490:	learn: 0.2425378	total: 3.96s	remaining: 4.1s
491:	learn: 0.2424272	total: 3.97s	remaining: 4.09s
492:	learn: 0.2423464	total: 3.97s	remaining: 4.09s
493:	learn: 0.2422399	total: 3.98s	remaining: 4.08s
494:	learn: 0.2421799	total: 4s	remaining: 4.08s
495:	learn: 0.2420730	total: 4.01s	remaining: 4.08s
496:	learn: 0.2420028	total: 4.02s	remaining: 4.07s
497:	learn: 0.2418598	total: 4.03s	remaining: 4.06s
498:	learn: 0.2417149	total: 4.03s	remaining: 4.05s
499:	learn: 0.2415966	total: 4.04s	remaining: 4.04s
500:	learn: 0.2414307	total: 4.05s	remaining: 4.03s
501:	learn: 0.2413622	total: 4.06s	remaining: 4.03s
502:	learn: 0.2412056	total: 4.07s	remaining: 4.02s
503:	learn: 0.2410850	total: 4.07s	remaining: 4.01s
504:	learn: 0.2410126	total: 4.08s	remaining: 4s
505:	learn: 0.2409193	total: 4.09s	remaining: 3.99s
506:	learn: 0.2407779	total: 4.09s	remaining: 3.98s
507:	learn: 0.2406883	total: 4.1s	remaining: 3.97s
508:	learn: 0.2406144	total: 4.11s	remaining: 3.96s
509:	learn: 0.2405453	total: 4.12s	remaining: 3.96s
510:	learn: 0.2404668	total: 4.13s	remaining: 3.95s
511:	learn: 0.2403382	total: 4.13s	remaining: 3.94s
512:	learn: 0.2402184	total: 4.14s	remaining: 3.93s
513:	learn: 0.2401061	total: 4.15s	remaining: 3.92s
514:	learn: 0.2399639	total: 4.16s	remaining: 3.92s
515:	learn: 0.2398331	total: 4.17s	remaining: 3.91s
516:	learn: 0.2397196	total: 4.17s	remaining: 3.9s
517:	learn: 0.2396206	total: 4.18s	remaining: 3.89s
518:	learn: 0.2395258	total: 4.19s	remaining: 3.88s
519:	learn: 0.2394617	total: 4.2s	remaining: 3.87s
520:	learn: 0.2393255	total: 4.21s	remaining: 3.87s
521:	learn: 0.2392424	total: 4.21s	remaining: 3.86s
522:	learn: 0.2391117	total: 4.22s	remaining: 3.85s
523:	learn: 0.2390098	total: 4.23s	remaining: 3.84s
524:	learn: 0.2389095	total: 4.24s	remaining: 3.83s
525:	learn: 0.2387522	total: 4.24s	remaining: 3.82s
526:	learn: 0.2386460	total: 4.25s	remaining: 3.82s
527:	learn: 0.2385157	total: 4.26s	remaining: 3.81s
528:	learn: 0.2384005	total: 4.27s	remaining: 3.8s
529:	learn: 0.2383404	total: 4.27s	remaining: 3.79s
530:	learn: 0.2382676	total: 4.28s	remaining: 3.78s
531:	learn: 0.2381505	total: 4.29s	remaining: 3.77s
532:	learn: 0.2380426	total: 4.3s	remaining: 3.77s
533:	learn: 0.2379516	total: 4.3s	remaining: 3.76s
534:	learn: 0.2378672	total: 4.31s	remaining: 3.75s
535:	learn: 0.2377842	total: 4.32s	remaining: 3.74s
536:	learn: 0.2377168	total: 4.33s	remaining: 3.73s
537:	learn: 0.2376473	total: 4.34s	remaining: 3.72s
538:	learn: 0.2375478	total: 4.34s	remaining: 3.72s
539:	learn: 0.2374676	total: 4.35s	remaining: 3.71s
540:	learn: 0.2373656	total: 4.36s	remaining: 3.7s
541:	learn: 0.2372403	total: 4.37s	remaining: 3.69s
542:	learn: 0.2371807	total: 4.38s	remaining: 3.68s
543:	learn: 0.2371210	total: 4.38s	remaining: 3.67s
544:	learn: 0.2370323	total: 4.39s	remaining: 3.67s
545:	learn: 0.2369225	total: 4.4s	remaining: 3.66s
546:	learn: 0.2368419	total: 4.41s	remaining: 3.65s
547:	learn: 0.2367849	total: 4.42s	remaining: 3.64s
548:	learn: 0.2366631	total: 4.43s	remaining: 3.63s
549:	learn: 0.2365313	total: 4.43s	remaining: 3.63s
550:	learn: 0.2364329	total: 4.44s	remaining: 3.62s
551:	learn: 0.2363669	total: 4.45s	remaining: 3.61s
552:	learn: 0.2362535	total: 4.46s	remaining: 3.6s
553:	learn: 0.2361556	total: 4.47s	remaining: 3.6s
554:	learn: 0.2360257	total: 4.47s	remaining: 3.59s
555:	learn: 0.2358791	total: 4.48s	remaining: 3.58s
556:	learn: 0.2357994	total: 4.49s	remaining: 3.57s
557:	learn: 0.2357139	total: 4.5s	remaining: 3.56s
558:	learn: 0.2355981	total: 4.51s	remaining: 3.56s
559:	learn: 0.2354827	total: 4.51s	remaining: 3.55s
560:	learn: 0.2353845	total: 4.52s	remaining: 3.54s
561:	learn: 0.2353248	total: 4.53s	remaining: 3.53s
562:	learn: 0.2352421	total: 4.54s	remaining: 3.52s
563:	learn: 0.2351725	total: 4.55s	remaining: 3.51s
564:	learn: 0.2351462	total: 4.55s	remaining: 3.51s
565:	learn: 0.2349939	total: 4.56s	remaining: 3.5s
566:	learn: 0.2349042	total: 4.57s	remaining: 3.49s
567:	learn: 0.2348001	total: 4.58s	remaining: 3.48s
568:	learn: 0.2347036	total: 4.58s	remaining: 3.47s
569:	learn: 0.2346163	total: 4.59s	remaining: 3.46s
570:	learn: 0.2345250	total: 4.6s	remaining: 3.46s
571:	learn: 0.2344409	total: 4.61s	remaining: 3.45s
572:	learn: 0.2343353	total: 4.62s	remaining: 3.44s
573:	learn: 0.2342590	total: 4.62s	remaining: 3.43s
574:	learn: 0.2341472	total: 4.63s	remaining: 3.42s
575:	learn: 0.2340437	total: 4.64s	remaining: 3.41s
576:	learn: 0.2339859	total: 4.64s	remaining: 3.4s
577:	learn: 0.2338970	total: 4.65s	remaining: 3.4s
578:	learn: 0.2338047	total: 4.66s	remaining: 3.39s
579:	learn: 0.2337383	total: 4.67s	remaining: 3.38s
580:	learn: 0.2336567	total: 4.68s	remaining: 3.37s
581:	learn: 0.2335831	total: 4.68s	remaining: 3.36s
582:	learn: 0.2334758	total: 4.69s	remaining: 3.35s
583:	learn: 0.2333985	total: 4.7s	remaining: 3.35s
584:	learn: 0.2332614	total: 4.71s	remaining: 3.34s
585:	learn: 0.2331571	total: 4.72s	remaining: 3.33s
586:	learn: 0.2330731	total: 4.72s	remaining: 3.32s
587:	learn: 0.2329887	total: 4.73s	remaining: 3.32s
588:	learn: 0.2328950	total: 4.74s	remaining: 3.31s
589:	learn: 0.2327972	total: 4.75s	remaining: 3.3s
590:	learn: 0.2327096	total: 4.76s	remaining: 3.29s
591:	learn: 0.2326265	total: 4.76s	remaining: 3.28s
592:	learn: 0.2325219	total: 4.77s	remaining: 3.27s
593:	learn: 0.2324530	total: 4.78s	remaining: 3.27s
594:	learn: 0.2323814	total: 4.79s	remaining: 3.26s
595:	learn: 0.2323273	total: 4.79s	remaining: 3.25s
596:	learn: 0.2322729	total: 4.8s	remaining: 3.24s
597:	learn: 0.2322046	total: 4.81s	remaining: 3.23s
598:	learn: 0.2321135	total: 4.82s	remaining: 3.23s
599:	learn: 0.2319944	total: 4.83s	remaining: 3.22s
600:	learn: 0.2319582	total: 4.83s	remaining: 3.21s
601:	learn: 0.2318835	total: 4.84s	remaining: 3.2s
602:	learn: 0.2318320	total: 4.85s	remaining: 3.19s
603:	learn: 0.2317575	total: 4.86s	remaining: 3.18s
604:	learn: 0.2316663	total: 4.87s	remaining: 3.18s
605:	learn: 0.2315737	total: 4.87s	remaining: 3.17s
606:	learn: 0.2314896	total: 4.88s	remaining: 3.16s
607:	learn: 0.2313679	total: 4.89s	remaining: 3.15s
608:	learn: 0.2312826	total: 4.9s	remaining: 3.14s
609:	learn: 0.2311869	total: 4.91s	remaining: 3.14s
610:	learn: 0.2310930	total: 4.91s	remaining: 3.13s
611:	learn: 0.2310152	total: 4.92s	remaining: 3.12s
612:	learn: 0.2308936	total: 4.93s	remaining: 3.11s
613:	learn: 0.2308012	total: 4.94s	remaining: 3.1s
614:	learn: 0.2306819	total: 4.95s	remaining: 3.1s
615:	learn: 0.2305891	total: 4.95s	remaining: 3.09s
616:	learn: 0.2304927	total: 4.96s	remaining: 3.08s
617:	learn: 0.2304434	total: 4.97s	remaining: 3.07s
618:	learn: 0.2303537	total: 4.98s	remaining: 3.06s
619:	learn: 0.2302542	total: 4.99s	remaining: 3.06s
620:	learn: 0.2301690	total: 5s	remaining: 3.05s
621:	learn: 0.2300825	total: 5.03s	remaining: 3.06s
622:	learn: 0.2299744	total: 5.04s	remaining: 3.05s
623:	learn: 0.2298453	total: 5.05s	remaining: 3.04s
624:	learn: 0.2297229	total: 5.06s	remaining: 3.03s
625:	learn: 0.2296675	total: 5.07s	remaining: 3.03s
626:	learn: 0.2296442	total: 5.07s	remaining: 3.02s
627:	learn: 0.2295763	total: 5.08s	remaining: 3.01s
628:	learn: 0.2294868	total: 5.09s	remaining: 3s
629:	learn: 0.2294090	total: 5.1s	remaining: 2.99s
630:	learn: 0.2293588	total: 5.11s	remaining: 2.98s
631:	learn: 0.2292471	total: 5.11s	remaining: 2.98s
632:	learn: 0.2291806	total: 5.12s	remaining: 2.97s
633:	learn: 0.2290827	total: 5.13s	remaining: 2.96s
634:	learn: 0.2290012	total: 5.14s	remaining: 2.95s
635:	learn: 0.2288757	total: 5.15s	remaining: 2.95s
636:	learn: 0.2288183	total: 5.16s	remaining: 2.94s
637:	learn: 0.2286849	total: 5.17s	remaining: 2.93s
638:	learn: 0.2285324	total: 5.17s	remaining: 2.92s
639:	learn: 0.2283907	total: 5.2s	remaining: 2.92s
640:	learn: 0.2283175	total: 5.21s	remaining: 2.92s
641:	learn: 0.2282168	total: 5.22s	remaining: 2.91s
642:	learn: 0.2281285	total: 5.22s	remaining: 2.9s
643:	learn: 0.2280420	total: 5.23s	remaining: 2.89s
644:	learn: 0.2279308	total: 5.24s	remaining: 2.88s
645:	learn: 0.2278534	total: 5.25s	remaining: 2.88s
646:	learn: 0.2278130	total: 5.26s	remaining: 2.87s
647:	learn: 0.2277102	total: 5.26s	remaining: 2.86s
648:	learn: 0.2276083	total: 5.27s	remaining: 2.85s
649:	learn: 0.2275155	total: 5.28s	remaining: 2.84s
650:	learn: 0.2274131	total: 5.29s	remaining: 2.83s
651:	learn: 0.2273420	total: 5.3s	remaining: 2.83s
652:	learn: 0.2272355	total: 5.3s	remaining: 2.82s
653:	learn: 0.2271202	total: 5.31s	remaining: 2.81s
654:	learn: 0.2270243	total: 5.32s	remaining: 2.8s
655:	learn: 0.2268846	total: 5.33s	remaining: 2.79s
656:	learn: 0.2267749	total: 5.34s	remaining: 2.79s
657:	learn: 0.2266967	total: 5.35s	remaining: 2.78s
658:	learn: 0.2265927	total: 5.35s	remaining: 2.77s
659:	learn: 0.2265394	total: 5.36s	remaining: 2.76s
660:	learn: 0.2264660	total: 5.37s	remaining: 2.75s
661:	learn: 0.2263446	total: 5.38s	remaining: 2.75s
662:	learn: 0.2262680	total: 5.38s	remaining: 2.74s
663:	learn: 0.2261824	total: 5.39s	remaining: 2.73s
664:	learn: 0.2261097	total: 5.4s	remaining: 2.72s
665:	learn: 0.2260224	total: 5.41s	remaining: 2.71s
666:	learn: 0.2259716	total: 5.42s	remaining: 2.7s
667:	learn: 0.2259430	total: 5.42s	remaining: 2.7s
668:	learn: 0.2257905	total: 5.43s	remaining: 2.69s
669:	learn: 0.2256938	total: 5.44s	remaining: 2.68s
670:	learn: 0.2256219	total: 5.45s	remaining: 2.67s
671:	learn: 0.2255524	total: 5.46s	remaining: 2.66s
672:	learn: 0.2254715	total: 5.46s	remaining: 2.65s
673:	learn: 0.2254158	total: 5.47s	remaining: 2.65s
674:	learn: 0.2253839	total: 5.48s	remaining: 2.64s
675:	learn: 0.2253013	total: 5.49s	remaining: 2.63s
676:	learn: 0.2251489	total: 5.5s	remaining: 2.62s
677:	learn: 0.2250699	total: 5.51s	remaining: 2.62s
678:	learn: 0.2250097	total: 5.51s	remaining: 2.61s
679:	learn: 0.2248665	total: 5.52s	remaining: 2.6s
680:	learn: 0.2247965	total: 5.53s	remaining: 2.59s
681:	learn: 0.2247411	total: 5.54s	remaining: 2.58s
682:	learn: 0.2246825	total: 5.55s	remaining: 2.58s
683:	learn: 0.2245700	total: 5.56s	remaining: 2.57s
684:	learn: 0.2245163	total: 5.56s	remaining: 2.56s
685:	learn: 0.2244668	total: 5.57s	remaining: 2.55s
686:	learn: 0.2243382	total: 5.58s	remaining: 2.54s
687:	learn: 0.2242659	total: 5.59s	remaining: 2.53s
688:	learn: 0.2241645	total: 5.6s	remaining: 2.53s
689:	learn: 0.2240620	total: 5.61s	remaining: 2.52s
690:	learn: 0.2239530	total: 5.62s	remaining: 2.51s
691:	learn: 0.2238558	total: 5.62s	remaining: 2.5s
692:	learn: 0.2238156	total: 5.63s	remaining: 2.49s
693:	learn: 0.2237333	total: 5.64s	remaining: 2.49s
694:	learn: 0.2236607	total: 5.65s	remaining: 2.48s
695:	learn: 0.2235790	total: 5.66s	remaining: 2.47s
696:	learn: 0.2234953	total: 5.66s	remaining: 2.46s
697:	learn: 0.2234194	total: 5.67s	remaining: 2.45s
698:	learn: 0.2233113	total: 5.68s	remaining: 2.45s
699:	learn: 0.2231717	total: 5.69s	remaining: 2.44s
700:	learn: 0.2230979	total: 5.7s	remaining: 2.43s
701:	learn: 0.2230280	total: 5.7s	remaining: 2.42s
702:	learn: 0.2229402	total: 5.71s	remaining: 2.41s
703:	learn: 0.2228697	total: 5.72s	remaining: 2.4s
704:	learn: 0.2228250	total: 5.73s	remaining: 2.4s
705:	learn: 0.2227600	total: 5.74s	remaining: 2.39s
706:	learn: 0.2226959	total: 5.75s	remaining: 2.38s
707:	learn: 0.2226225	total: 5.75s	remaining: 2.37s
708:	learn: 0.2225700	total: 5.76s	remaining: 2.36s
709:	learn: 0.2224808	total: 5.77s	remaining: 2.36s
710:	learn: 0.2224083	total: 5.78s	remaining: 2.35s
711:	learn: 0.2223819	total: 5.79s	remaining: 2.34s
712:	learn: 0.2222774	total: 5.79s	remaining: 2.33s
713:	learn: 0.2221774	total: 5.8s	remaining: 2.32s
714:	learn: 0.2221404	total: 5.81s	remaining: 2.31s
715:	learn: 0.2220737	total: 5.82s	remaining: 2.31s
716:	learn: 0.2219876	total: 5.83s	remaining: 2.3s
717:	learn: 0.2218826	total: 5.83s	remaining: 2.29s
718:	learn: 0.2218162	total: 5.84s	remaining: 2.28s
719:	learn: 0.2217588	total: 5.85s	remaining: 2.27s
720:	learn: 0.2216743	total: 5.86s	remaining: 2.27s
721:	learn: 0.2215722	total: 5.87s	remaining: 2.26s
722:	learn: 0.2214773	total: 5.87s	remaining: 2.25s
723:	learn: 0.2214031	total: 5.88s	remaining: 2.24s
724:	learn: 0.2213533	total: 5.89s	remaining: 2.23s
725:	learn: 0.2212922	total: 5.9s	remaining: 2.23s
726:	learn: 0.2212624	total: 5.9s	remaining: 2.22s
727:	learn: 0.2211598	total: 5.91s	remaining: 2.21s
728:	learn: 0.2210599	total: 5.92s	remaining: 2.2s
729:	learn: 0.2210023	total: 5.93s	remaining: 2.19s
730:	learn: 0.2209159	total: 5.94s	remaining: 2.18s
731:	learn: 0.2208022	total: 5.95s	remaining: 2.18s
732:	learn: 0.2206707	total: 5.95s	remaining: 2.17s
733:	learn: 0.2205867	total: 5.96s	remaining: 2.16s
734:	learn: 0.2205174	total: 5.97s	remaining: 2.15s
735:	learn: 0.2204280	total: 5.98s	remaining: 2.14s
736:	learn: 0.2203741	total: 5.99s	remaining: 2.14s
737:	learn: 0.2203094	total: 5.99s	remaining: 2.13s
738:	learn: 0.2202555	total: 6s	remaining: 2.12s
739:	learn: 0.2201774	total: 6.01s	remaining: 2.11s
740:	learn: 0.2201078	total: 6.04s	remaining: 2.11s
741:	learn: 0.2200129	total: 6.04s	remaining: 2.1s
742:	learn: 0.2199291	total: 6.05s	remaining: 2.09s
743:	learn: 0.2198576	total: 6.06s	remaining: 2.08s
744:	learn: 0.2198044	total: 6.07s	remaining: 2.08s
745:	learn: 0.2196979	total: 6.08s	remaining: 2.07s
746:	learn: 0.2195756	total: 6.08s	remaining: 2.06s
747:	learn: 0.2194931	total: 6.09s	remaining: 2.05s
748:	learn: 0.2194525	total: 6.1s	remaining: 2.04s
749:	learn: 0.2194016	total: 6.11s	remaining: 2.04s
750:	learn: 0.2193025	total: 6.12s	remaining: 2.03s
751:	learn: 0.2192046	total: 6.12s	remaining: 2.02s
752:	learn: 0.2191301	total: 6.13s	remaining: 2.01s
753:	learn: 0.2190905	total: 6.14s	remaining: 2s
754:	learn: 0.2189554	total: 6.15s	remaining: 1.99s
755:	learn: 0.2189033	total: 6.16s	remaining: 1.99s
756:	learn: 0.2188222	total: 6.16s	remaining: 1.98s
757:	learn: 0.2187622	total: 6.17s	remaining: 1.97s
758:	learn: 0.2186578	total: 6.18s	remaining: 1.96s
759:	learn: 0.2185941	total: 6.19s	remaining: 1.95s
760:	learn: 0.2184803	total: 6.2s	remaining: 1.95s
761:	learn: 0.2184085	total: 6.2s	remaining: 1.94s
762:	learn: 0.2183041	total: 6.21s	remaining: 1.93s
763:	learn: 0.2181917	total: 6.22s	remaining: 1.92s
764:	learn: 0.2180963	total: 6.23s	remaining: 1.91s
765:	learn: 0.2180173	total: 6.24s	remaining: 1.9s
766:	learn: 0.2179318	total: 6.24s	remaining: 1.9s
767:	learn: 0.2178655	total: 6.25s	remaining: 1.89s
768:	learn: 0.2177910	total: 6.26s	remaining: 1.88s
769:	learn: 0.2176954	total: 6.27s	remaining: 1.87s
770:	learn: 0.2175535	total: 6.28s	remaining: 1.86s
771:	learn: 0.2174552	total: 6.28s	remaining: 1.85s
772:	learn: 0.2173661	total: 6.29s	remaining: 1.85s
773:	learn: 0.2173317	total: 6.3s	remaining: 1.84s
774:	learn: 0.2172570	total: 6.31s	remaining: 1.83s
775:	learn: 0.2171970	total: 6.32s	remaining: 1.82s
776:	learn: 0.2171292	total: 6.32s	remaining: 1.81s
777:	learn: 0.2170468	total: 6.33s	remaining: 1.81s
778:	learn: 0.2169581	total: 6.34s	remaining: 1.8s
779:	learn: 0.2168664	total: 6.35s	remaining: 1.79s
780:	learn: 0.2168285	total: 6.36s	remaining: 1.78s
781:	learn: 0.2167979	total: 6.36s	remaining: 1.77s
782:	learn: 0.2167352	total: 6.37s	remaining: 1.77s
783:	learn: 0.2166256	total: 6.38s	remaining: 1.76s
784:	learn: 0.2165272	total: 6.39s	remaining: 1.75s
785:	learn: 0.2164232	total: 6.4s	remaining: 1.74s
786:	learn: 0.2163554	total: 6.41s	remaining: 1.73s
787:	learn: 0.2163180	total: 6.41s	remaining: 1.73s
788:	learn: 0.2162515	total: 6.42s	remaining: 1.72s
789:	learn: 0.2162104	total: 6.43s	remaining: 1.71s
790:	learn: 0.2161252	total: 6.44s	remaining: 1.7s
791:	learn: 0.2160557	total: 6.45s	remaining: 1.69s
792:	learn: 0.2159999	total: 6.45s	remaining: 1.68s
793:	learn: 0.2159272	total: 6.46s	remaining: 1.68s
794:	learn: 0.2158322	total: 6.47s	remaining: 1.67s
795:	learn: 0.2157673	total: 6.48s	remaining: 1.66s
796:	learn: 0.2156839	total: 6.49s	remaining: 1.65s
797:	learn: 0.2155619	total: 6.49s	remaining: 1.64s
798:	learn: 0.2154489	total: 6.5s	remaining: 1.64s
799:	learn: 0.2153295	total: 6.51s	remaining: 1.63s
800:	learn: 0.2152584	total: 6.52s	remaining: 1.62s
801:	learn: 0.2151861	total: 6.53s	remaining: 1.61s
802:	learn: 0.2150888	total: 6.54s	remaining: 1.6s
803:	learn: 0.2150363	total: 6.54s	remaining: 1.59s
804:	learn: 0.2149570	total: 6.55s	remaining: 1.59s
805:	learn: 0.2149127	total: 6.56s	remaining: 1.58s
806:	learn: 0.2148487	total: 6.57s	remaining: 1.57s
807:	learn: 0.2147976	total: 6.58s	remaining: 1.56s
808:	learn: 0.2147059	total: 6.58s	remaining: 1.55s
809:	learn: 0.2146223	total: 6.59s	remaining: 1.55s
810:	learn: 0.2145257	total: 6.6s	remaining: 1.54s
811:	learn: 0.2144414	total: 6.61s	remaining: 1.53s
812:	learn: 0.2143711	total: 6.62s	remaining: 1.52s
813:	learn: 0.2143174	total: 6.62s	remaining: 1.51s
814:	learn: 0.2142902	total: 6.63s	remaining: 1.5s
815:	learn: 0.2141837	total: 6.64s	remaining: 1.5s
816:	learn: 0.2141463	total: 6.65s	remaining: 1.49s
817:	learn: 0.2140290	total: 6.66s	remaining: 1.48s
818:	learn: 0.2139203	total: 6.67s	remaining: 1.47s
819:	learn: 0.2138581	total: 6.67s	remaining: 1.46s
820:	learn: 0.2137948	total: 6.68s	remaining: 1.46s
821:	learn: 0.2137465	total: 6.69s	remaining: 1.45s
822:	learn: 0.2136808	total: 6.7s	remaining: 1.44s
823:	learn: 0.2135773	total: 6.71s	remaining: 1.43s
824:	learn: 0.2135079	total: 6.71s	remaining: 1.42s
825:	learn: 0.2134742	total: 6.72s	remaining: 1.42s
826:	learn: 0.2133768	total: 6.73s	remaining: 1.41s
827:	learn: 0.2132408	total: 6.74s	remaining: 1.4s
828:	learn: 0.2131785	total: 6.75s	remaining: 1.39s
829:	learn: 0.2130925	total: 6.75s	remaining: 1.38s
830:	learn: 0.2130297	total: 6.76s	remaining: 1.38s
831:	learn: 0.2129353	total: 6.77s	remaining: 1.37s
832:	learn: 0.2128608	total: 6.78s	remaining: 1.36s
833:	learn: 0.2128007	total: 6.78s	remaining: 1.35s
834:	learn: 0.2127317	total: 6.79s	remaining: 1.34s
835:	learn: 0.2126471	total: 6.8s	remaining: 1.33s
836:	learn: 0.2125967	total: 6.81s	remaining: 1.33s
837:	learn: 0.2125334	total: 6.82s	remaining: 1.32s
838:	learn: 0.2124508	total: 6.82s	remaining: 1.31s
839:	learn: 0.2123734	total: 6.83s	remaining: 1.3s
840:	learn: 0.2123506	total: 6.84s	remaining: 1.29s
841:	learn: 0.2122618	total: 6.85s	remaining: 1.28s
842:	learn: 0.2121757	total: 6.86s	remaining: 1.28s
843:	learn: 0.2120804	total: 6.87s	remaining: 1.27s
844:	learn: 0.2119854	total: 6.87s	remaining: 1.26s
845:	learn: 0.2119050	total: 6.88s	remaining: 1.25s
846:	learn: 0.2118468	total: 6.89s	remaining: 1.24s
847:	learn: 0.2117937	total: 6.9s	remaining: 1.24s
848:	learn: 0.2117019	total: 6.91s	remaining: 1.23s
849:	learn: 0.2116323	total: 6.91s	remaining: 1.22s
850:	learn: 0.2115975	total: 6.92s	remaining: 1.21s
851:	learn: 0.2115077	total: 6.93s	remaining: 1.2s
852:	learn: 0.2114678	total: 6.94s	remaining: 1.2s
853:	learn: 0.2114577	total: 6.95s	remaining: 1.19s
854:	learn: 0.2114216	total: 6.95s	remaining: 1.18s
855:	learn: 0.2113342	total: 6.96s	remaining: 1.17s
856:	learn: 0.2112600	total: 6.97s	remaining: 1.16s
857:	learn: 0.2112082	total: 6.98s	remaining: 1.15s
858:	learn: 0.2111481	total: 6.99s	remaining: 1.15s
859:	learn: 0.2110980	total: 6.99s	remaining: 1.14s
860:	learn: 0.2110289	total: 7s	remaining: 1.13s
861:	learn: 0.2109409	total: 7.01s	remaining: 1.12s
862:	learn: 0.2108535	total: 7.02s	remaining: 1.11s
863:	learn: 0.2107722	total: 7.05s	remaining: 1.11s
864:	learn: 0.2107075	total: 7.05s	remaining: 1.1s
865:	learn: 0.2106164	total: 7.06s	remaining: 1.09s
866:	learn: 0.2105748	total: 7.07s	remaining: 1.08s
867:	learn: 0.2104988	total: 7.08s	remaining: 1.08s
868:	learn: 0.2104088	total: 7.09s	remaining: 1.07s
869:	learn: 0.2103383	total: 7.1s	remaining: 1.06s
870:	learn: 0.2102361	total: 7.11s	remaining: 1.05s
871:	learn: 0.2101613	total: 7.11s	remaining: 1.04s
872:	learn: 0.2101200	total: 7.12s	remaining: 1.03s
873:	learn: 0.2100064	total: 7.13s	remaining: 1.03s
874:	learn: 0.2099326	total: 7.14s	remaining: 1.02s
875:	learn: 0.2098829	total: 7.14s	remaining: 1.01s
876:	learn: 0.2097791	total: 7.15s	remaining: 1s
877:	learn: 0.2096928	total: 7.16s	remaining: 995ms
878:	learn: 0.2096116	total: 7.17s	remaining: 987ms
879:	learn: 0.2095156	total: 7.18s	remaining: 979ms
880:	learn: 0.2094534	total: 7.19s	remaining: 971ms
881:	learn: 0.2094165	total: 7.19s	remaining: 963ms
882:	learn: 0.2093276	total: 7.2s	remaining: 954ms
883:	learn: 0.2092722	total: 7.21s	remaining: 946ms
884:	learn: 0.2092006	total: 7.22s	remaining: 938ms
885:	learn: 0.2091460	total: 7.23s	remaining: 930ms
886:	learn: 0.2090897	total: 7.24s	remaining: 922ms
887:	learn: 0.2090399	total: 7.24s	remaining: 914ms
888:	learn: 0.2090023	total: 7.25s	remaining: 905ms
889:	learn: 0.2089218	total: 7.26s	remaining: 897ms
890:	learn: 0.2087982	total: 7.27s	remaining: 889ms
891:	learn: 0.2087321	total: 7.28s	remaining: 881ms
892:	learn: 0.2086852	total: 7.28s	remaining: 873ms
893:	learn: 0.2085606	total: 7.29s	remaining: 865ms
894:	learn: 0.2084904	total: 7.3s	remaining: 856ms
895:	learn: 0.2084076	total: 7.31s	remaining: 848ms
896:	learn: 0.2083522	total: 7.32s	remaining: 840ms
897:	learn: 0.2082683	total: 7.32s	remaining: 832ms
898:	learn: 0.2082327	total: 7.33s	remaining: 824ms
899:	learn: 0.2081190	total: 7.34s	remaining: 816ms
900:	learn: 0.2080857	total: 7.35s	remaining: 808ms
901:	learn: 0.2080346	total: 7.36s	remaining: 799ms
902:	learn: 0.2079675	total: 7.37s	remaining: 791ms
903:	learn: 0.2078992	total: 7.37s	remaining: 783ms
904:	learn: 0.2078654	total: 7.38s	remaining: 775ms
905:	learn: 0.2078071	total: 7.39s	remaining: 767ms
906:	learn: 0.2077478	total: 7.4s	remaining: 758ms
907:	learn: 0.2076774	total: 7.41s	remaining: 750ms
908:	learn: 0.2075978	total: 7.41s	remaining: 742ms
909:	learn: 0.2075337	total: 7.42s	remaining: 734ms
910:	learn: 0.2074656	total: 7.43s	remaining: 726ms
911:	learn: 0.2074219	total: 7.44s	remaining: 718ms
912:	learn: 0.2073472	total: 7.44s	remaining: 709ms
913:	learn: 0.2073021	total: 7.45s	remaining: 701ms
914:	learn: 0.2071893	total: 7.46s	remaining: 693ms
915:	learn: 0.2071129	total: 7.47s	remaining: 685ms
916:	learn: 0.2069858	total: 7.48s	remaining: 677ms
917:	learn: 0.2068877	total: 7.49s	remaining: 669ms
918:	learn: 0.2068143	total: 7.49s	remaining: 660ms
919:	learn: 0.2067553	total: 7.5s	remaining: 652ms
920:	learn: 0.2067146	total: 7.51s	remaining: 644ms
921:	learn: 0.2066385	total: 7.52s	remaining: 636ms
922:	learn: 0.2065674	total: 7.53s	remaining: 628ms
923:	learn: 0.2064703	total: 7.54s	remaining: 620ms
924:	learn: 0.2063905	total: 7.54s	remaining: 612ms
925:	learn: 0.2063215	total: 7.55s	remaining: 604ms
926:	learn: 0.2062382	total: 7.56s	remaining: 595ms
927:	learn: 0.2061738	total: 7.57s	remaining: 587ms
928:	learn: 0.2061088	total: 7.58s	remaining: 579ms
929:	learn: 0.2060524	total: 7.58s	remaining: 571ms
930:	learn: 0.2060114	total: 7.59s	remaining: 563ms
931:	learn: 0.2059788	total: 7.6s	remaining: 555ms
932:	learn: 0.2058946	total: 7.61s	remaining: 547ms
933:	learn: 0.2058709	total: 7.62s	remaining: 538ms
934:	learn: 0.2058046	total: 7.63s	remaining: 530ms
935:	learn: 0.2057623	total: 7.64s	remaining: 522ms
936:	learn: 0.2057199	total: 7.64s	remaining: 514ms
937:	learn: 0.2056364	total: 7.65s	remaining: 506ms
938:	learn: 0.2055876	total: 7.66s	remaining: 498ms
939:	learn: 0.2055335	total: 7.67s	remaining: 490ms
940:	learn: 0.2054965	total: 7.68s	remaining: 481ms
941:	learn: 0.2054202	total: 7.69s	remaining: 473ms
942:	learn: 0.2053639	total: 7.69s	remaining: 465ms
943:	learn: 0.2052773	total: 7.7s	remaining: 457ms
944:	learn: 0.2051818	total: 7.71s	remaining: 449ms
945:	learn: 0.2051274	total: 7.72s	remaining: 441ms
946:	learn: 0.2050498	total: 7.73s	remaining: 432ms
947:	learn: 0.2049594	total: 7.73s	remaining: 424ms
948:	learn: 0.2049226	total: 7.74s	remaining: 416ms
949:	learn: 0.2048216	total: 7.75s	remaining: 408ms
950:	learn: 0.2047653	total: 7.76s	remaining: 400ms
951:	learn: 0.2047214	total: 7.77s	remaining: 392ms
952:	learn: 0.2046157	total: 7.78s	remaining: 383ms
953:	learn: 0.2045532	total: 7.78s	remaining: 375ms
954:	learn: 0.2044984	total: 7.79s	remaining: 367ms
955:	learn: 0.2044390	total: 7.8s	remaining: 359ms
956:	learn: 0.2043388	total: 7.81s	remaining: 351ms
957:	learn: 0.2042348	total: 7.82s	remaining: 343ms
958:	learn: 0.2041639	total: 7.82s	remaining: 335ms
959:	learn: 0.2041364	total: 7.83s	remaining: 326ms
960:	learn: 0.2040939	total: 7.84s	remaining: 318ms
961:	learn: 0.2040458	total: 7.85s	remaining: 310ms
962:	learn: 0.2040047	total: 7.86s	remaining: 302ms
963:	learn: 0.2039391	total: 7.86s	remaining: 294ms
964:	learn: 0.2038321	total: 7.87s	remaining: 286ms
965:	learn: 0.2037040	total: 7.88s	remaining: 277ms
966:	learn: 0.2036320	total: 7.89s	remaining: 269ms
967:	learn: 0.2035491	total: 7.9s	remaining: 261ms
968:	learn: 0.2034893	total: 7.9s	remaining: 253ms
969:	learn: 0.2034253	total: 7.91s	remaining: 245ms
970:	learn: 0.2033788	total: 7.92s	remaining: 237ms
971:	learn: 0.2033201	total: 7.93s	remaining: 228ms
972:	learn: 0.2032421	total: 7.94s	remaining: 220ms
973:	learn: 0.2031651	total: 7.95s	remaining: 212ms
974:	learn: 0.2030931	total: 7.95s	remaining: 204ms
975:	learn: 0.2030350	total: 7.96s	remaining: 196ms
976:	learn: 0.2029410	total: 7.97s	remaining: 188ms
977:	learn: 0.2028739	total: 7.98s	remaining: 179ms
978:	learn: 0.2028486	total: 7.99s	remaining: 171ms
979:	learn: 0.2028286	total: 7.99s	remaining: 163ms
980:	learn: 0.2027439	total: 8s	remaining: 155ms
981:	learn: 0.2026671	total: 8.01s	remaining: 147ms
982:	learn: 0.2026179	total: 8.02s	remaining: 139ms
983:	learn: 0.2025590	total: 8.03s	remaining: 131ms
984:	learn: 0.2024847	total: 8.03s	remaining: 122ms
985:	learn: 0.2024418	total: 8.04s	remaining: 114ms
986:	learn: 0.2023259	total: 8.08s	remaining: 106ms
987:	learn: 0.2022827	total: 8.09s	remaining: 98.2ms
988:	learn: 0.2021969	total: 8.09s	remaining: 90ms
989:	learn: 0.2020952	total: 8.1s	remaining: 81.8ms
990:	learn: 0.2020277	total: 8.11s	remaining: 73.7ms
991:	learn: 0.2019527	total: 8.12s	remaining: 65.5ms
992:	learn: 0.2018579	total: 8.13s	remaining: 57.3ms
993:	learn: 0.2018068	total: 8.13s	remaining: 49.1ms
994:	learn: 0.2017433	total: 8.14s	remaining: 40.9ms
995:	learn: 0.2016975	total: 8.15s	remaining: 32.7ms
996:	learn: 0.2016517	total: 8.16s	remaining: 24.6ms
997:	learn: 0.2015688	total: 8.17s	remaining: 16.4ms
998:	learn: 0.2014925	total: 8.17s	remaining: 8.18ms
999:	learn: 0.2014498	total: 8.18s	remaining: 0us

#predicting
catboost_predictions = catboost_model.predict(X_test)

metrics = ['Precision', 'Recall', 'F1', 'AUC','Accuracy']
eval_metrics = catboost_model.eval_metrics(train_pool,
                                           metrics=metrics,
                                           plot=True)

MetricVisualizer(layout=Layout(align_self='stretch', height='500px'))

As shown in the charts of our model we have quite the impressive metrics that we will be discussing further on in our evalutation but here is an overvie for now

metrics_df = pd.DataFrame({'model':[],'Precision':[], 'Recall':[], 'F1':[], 'AUC':[],'Accuracy':[]})

catboost_metrics = {
        'Model':[],
         'Precision':[],
                  'Recall':[],
                  'F1':[], 
                  'AUC':[],
                  'Accuracy':[]
                 }
catboost_metrics['Model']= 'CatBoost'
metrics = ['Precision', 'Recall', 'F1', 'AUC','Accuracy']
for metric in metrics:
    catboost_metrics[metric].append(np.mean(eval_metrics[metric]))
    print(str(metric)+": {}".format(np.mean(eval_metrics[metric])))

Precision: 0.9604845086979267
Recall: 0.11233274647887324
F1: 0.18988474995917884
AUC: 0.8355849401675709
Accuracy: 0.9133412127440904

Now let's try building other models to see the overall performance of other models compared to our main model and for that we will be using a variety of classifiers.
We will create a function that will automate the learning and the predictions of all of the models and returns a dataframe containing the evaluation metrics

def model(models,X_train, y_train,X_test,y_test):
    
    metrics = {'Model': ['RandomForestClassifier','XGBClassifier','lgb.LGBMClassifier','SVC','LogisticRegression','SGDClassifier'],
                  'Precision':[],
                  'Recall':[],
                  'F1':[], 
                  'AUC':[],
                  'Accuracy':[]
                 }
 
    for mod in models:
        model = mod
        model.fit(X_train, y_train)
        predictions = model.predict(X_test)  
        metrics['Precision'].append(precision_score(y_test,predictions))
        metrics['Recall'].append(recall_score(y_test, predictions))
        metrics['F1'].append(f1_score(y_test, predictions))
        metrics['AUC'].append(roc_auc_score(y_test, predictions))
        metrics['Accuracy'].append(accuracy_score(y_test, predictions))
        

    metrics = pd.DataFrame(metrics)
    return metrics
        

models = [RandomForestClassifier(n_estimators=1000),XGBClassifier(),lgb.LGBMClassifier(learning_rate=0.09,max_depth=-5,random_state=42),SVC(),LogisticRegression(),SGDClassifier()]

metrics = model(models,X_train, y_train,X_test,y_test)

metrics_df = metrics.append(pd.DataFrame(catboost_metrics), ignore_index=True)

metrics_df.sort_values(by=['Accuracy'],ascending=False)

	Model	Precision	Recall	F1	AUC	Accuracy
6	CatBoost	0.960485	0.112333	0.189885	0.835585	0.913341
2	lgb.LGBMClassifier	0.000000	0.000000	0.000000	0.500000	0.904452
3	SVC	0.000000	0.000000	0.000000	0.500000	0.904452
1	XGBClassifier	0.000000	0.000000	0.000000	0.498864	0.902397
0	RandomForestClassifier	0.101562	0.465950	0.166774	0.515254	0.555137
4	LogisticRegression	0.096712	0.537634	0.163934	0.503577	0.476027
5	SGDClassifier	0.095405	0.878136	0.172111	0.499273	0.192808

CatBoost confusion matrix

from sklearn.metrics import confusion_matrix
cf_matrix = confusion_matrix(y_test, catboost_predictions)
sns.heatmap(cf_matrix/np.sum(cf_matrix), annot=True, 
            fmt='.2%', cmap='Blues')

<AxesSubplot:>

Looking at the dataframe we can clearly notice high accuracies ranging from 0.9 to 0.91 in 5 of the classifiers with a huge fall to the last classifier with a value of 0.19. the overall accuracy we have found describes the good performace of the model yet it's not enough for a full evaluation.
We need to look also at the Precision which indicates whether classifies negative values as positive.
for the Light GBM ,SVM XGboost we have a 0% precision with 90% accuracy ! This is dangerous if not noticed. and this means that out of all the positive values non are predicted correctly. So they have an overall high accuracy but for the 10% of our test dataset they are predicted falsely and for this reason these models are eliminated.

Now looking at the CatBoost model we can identify a harmonic balance between the metrics with an accuracy of 91% and precision of 96% ! and this indicates clearly that our model is ROBUST. it predicted with a high precision the positive values and of course with a high accuracy the negative values. as for the AUC we have a value of 83 that gives an idea about the overall accuracy of the predicted values with the area under the curve which is higher than that above the curve.

For those reasons. we will be choosing the CatBoost model as the victorious among the others and the one we will use to deploy our app and give insights to the client.

Deployment

Let's export our model in order to build a web application and give insights to our client

import pickle
# open a file, where you ant to store the data
file = open('cat_boost_regression_model.pkl', 'wb')

# dump information to that file
pickle.dump(catboost_model, file)

Unpublished Work © 2022 Nizar Ben Hmida

Email : nizar.benhmida@esprit.tn
LinkedIn : https://www.linkedin.com/in/nizar-ben-hmida-76480b164/